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PREFACE: THE TLT GROUP AND THE FLASHLIGHT PROGRAM 


The Teaching, Learning, and Technology Group, a not-for-profit company, helps institutions improve teaching and 
learning with technology. The TLT Group provides assessment tools, materials, online and on-site training, and 
consulting to help clarify goals and strategies, assemble a balanced portfolio of educational improvements, deal with 
the support service crisis, rethink courses of study, improve teaching in individual courses. In addition to its 
institutional subscriptions, online workshops, and consulting services, the TLT Group also provides a wide range of 
free materials and webcasts. 

The TLT Group’s Flashlight™ Program is the largest and best-known program in the world to help educators assess 
educational uses of technology. Flashlight provides training, consulting (including external evaluations), and 
evaluation tool kits. This Cost Analysis Handbook is one of over 20 tool kits, study packages, rubrics and tutorials 
produced and distributed by the Flashlight Program. Most Flashlight resources are available only to subscribing 
institutions, which receive site licenses. Subscribing institutions receive a site license for this book, for example, 
which means they can legally make copies of this book for their own staff and students. Individuals may buy copies 
of this book from The TLT Group, but are not permitted to make copies of the book. 

Flashlight also offers a wide range of free articles, case studies, slideshows, and other resources. For example, this 
program has a free e-newsletter, F-LIGHT, published approximately every six weeks, which features case studies of 
successful studies and news from the Flashlight Program. For a free subscription, send e-mail to 
listproc@listproc.wsu.edu with no subject header. The message should say “SUBSCRIBE F-LIGHT yourfirstname 
yourlastname”. 

To order additional copies of this publication or for information about Flashlight tools and services, visit our web site at 
www.tltgroup.org or send a message to flashlight@tltgroup.org. 


Copyright 1999, 2003 by the 
Corporation for Public Broadcasting 
Published by 
The TLT Group 
One Columbia Avenue 
Takoma Park, MD 20912 USA 
Printed in the United States of America 
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INTRODUCTION 

There are several reasons to take a fresh look at the 
costs of using technology in education. 

1. Information about costs in higher education 
is relatively ambiguous. That goes for all 
aspects of education and includes costs of 
programs and services that use technology, 
or support its use. 

2. There is alarm on many campuses that 
technology costs are out of control. When 
is this true? Are there circumstances where 
technology use does help reallocate time, 
money or other resources? 

3. The rapid pace of technological change, and 
the resultant frequency of programmatic 
innovation, make past experience a less 
reliable guide, and increase the chances of 
cost over-runs and staff burnout, especially 
when technology use is rapidly expanding. 

4. In an era of national and state budget 
deficits, technology can be one of the first 
targets for cuts. 

Confusion about cost data 

Assessing the costs of educational activities using 
technology has never been easy. Over the past five 
years, there have been mounting debates about costs 
and prices in higher education. The National 
Commission on the Cost of Higher Education 
(Harvey et. al., 1998) recognized that financial 
information about higher education remains hidden 
"in a veil of obscurity." 

As a result of the Commission’s recommendations 
and the Higher Education Reauthorization Act of 
1998, Congress mandated a three-year College Cost 
Study. The recently released Study of College Costs 
and Prices, 1988 to 1997-98 , produced by the U.S. 
Department of Education’s National Center for 
Education Statistics, found “variations in the nature 
and the strength of relationships between costs and 
prices across type of institutions, and within types of 
institutions over time” (U.S. Dept of Ed, 2001, p. vi). 
Most significant are the variables of declining state 
appropriations, increases in instructional expenditures 
such as faculty salaries, and increasing reliance on 
institutional aid or tuition discounting. According to 
Gordon Winston in his commissioned paper for the 
Study, “prices never cover costs so every consumer is 
subsidized” and “Colleges are part charity and part 
commerce - churches and car dealers” 
(2001, pp. 117-118). These facts have at least two 
important consequences for cost studies: 


a) studies about the expense or break-even 
possibilities of programs in one institution or 
state do not mean that similar programs 
elsewhere will have similar costs or 
revenues, and 

b) cost studies will often need to deal with 
confusing questions about appropriate levels of 
subsidy. 

The National Association for College and University 
Business Officers (NACUBO) also responded to the 
Cost Commission report. In “Explaining College 
Costs: NACUBO’s Methodology for Identifying the 
Costs of Delivering Undergraduate Education” 
(2002), the Association reported on its widespread 
effort to develop a uniform methodology and conduct 
pilot tests. Pilot test results with 150 institutions 
reveal that, as predicted, “cost exceeds price” and 
that “the greatest costs are for instruction and student 
services.” The most important result from the 
NACUBO effort is a well-defined methodology and 
agreed-upon approach for measuring the cost of 
education at different types of institutions. This will 
allow for more refined and useful benchmarking, 
something long sought by NACUBO and others. 

Part of the problem, according to Hans Jenny in 
another NACUBO publication, is that “higher 
education accounting is not organized to answer 
questions concerning the full costs of teaching a 
conventional course, conducting a seminar, admitting 
a freshman class, or managing the institution’s 
heating and cooling system. . .Sometimes the normal 
accounting system is so far removed from what is 
needed that elaborate new stand-alone costing models 
must be constructed” (Jenny, 1996, p. 93). 

A growing group of institutions, including many 
within the Consortium of Liberal Arts Colleges and 
the Council of Independent Colleges, are working on 
another type of Costs Project. The study is driven by 
the perception that "Leaders of IT organizations are 
troubled by the lack of reliable benchmarks or 
comparative data on which to base decisions about 
support services" (Leach and Smallen, 1998, p. 38). 
The project has developed a comprehensive survey 
for submission by IT directors to help them pool data 
across institutions. More information is available at 
http://costsproiect.org . 

Decreasing IT budgets 

Academic computing budgets continue to decline, 
according to annual surveys conducted since 1990 by 
Kenneth Green’s Campus Computing Project. Over 
18.3% of respondents reported a decline in 2001, 
compared to 1 1.7% in 2000. This “comes at the end 
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of a seven year cycle in which campus technology 
budgets increased dramatically” (Green, 2001, p. 2). 

This change is driven in part by state budget crises 
across the nation. “Technology is on the chopping 
block,” and “states and institutions view technology 
spending as expendable,” reports Camevale (2002, p. 
A29). Green explains that the downturn in 
technology spending was predictable after seven 
years of growth; he points out that state funding is an 
“easy target in an economic downturn” and that 
“many elected officials believe that ‘technology had 
its turn’” (Green, 2002, pp. 46-47). 

Related efforts to study the costs 

OF EDUCATION 

Thousands of institutions across the world are trying 
new technologies, upgrading their computer labs, 
creating online courses, and examining the cost 
issues involved. While the Flashlight Program and 
this Handbook are intended to provide leadership in 
this arena, much can be learned from the more 
general higher education cost literature and from the 
many case studies and best practices about using 
emerging technologies. 

In particular, two sources are critical to readers who 
wish to understand the topic of costs on a basic level. 
The first source includes the publications of the 
National Association of College and University 
Business Officers (NACUBO), particularly Jenny’s 
(1996) A Cost Accounting Handbook for Colleges 
and Universities. Other classic NACUBO 
monographs for cost studies include A Cost 
Accounting Handbook for Colleges and Universities 
(Hyatt, 1973) and the NACUBO Handbook: College 
and University Budgeting (Meisinger, 1994). Another 
resource developed by a consortium of institutions 
and agencies in the United Kingdom is the 
Management Information for Decision-Making: 
Costing Guidelines for Higher Education Institutions 
(Joint Funding Councils, 1997). Both provide a 
sophisticated approach to costing and understanding 
the myriad of factors that must be considered, such as 
faculty workload and space utilization. They include 
numerous print worksheets, the latter with many 
electronic spreadsheets. 

Most interesting in these sets of resources is their 
presentation of a sample method for costing an 
individual course, which is sometimes a useful unit of 
analysis when looking at the implementation of 
technology. While Jenny presents a micro-costing, 
full cost, four-tier model of allocating expenditures 
and revenues to a specific course, the United 


Kingdom’s Joint Funding Councils study uses a 
hybrid of the activity-based costing model and one 
typically associated with indirect cost recovery. 
These terms will be defined later and are included in 
the Glossary in Appendix A. As readers learn more 
about costing and apply the specific Economic Model 
suggested in this Handbook , they will recognize an 
evolving and hybrid approach that takes assumptions 
and components from several different 
methodologies. 

This Handbook is unique, in that it helps the reader 
focus on developing a model that is unique to their 
use of technology and situation. Related efforts by 
NCHEMS, WICHE, and the Center for Academic 
Transformation promote specific models with 
detailed instructions for their completion. 

Other Efforts to Model the Costs of 
Educational Programs Using 
Technology 

The United Kingdom and Canada: The research 
study on the “Costs of Networked Learning” 
conducted by Paul Bacsich and others offers an 
important set of guidelines and models for 
approaching instructional technology costs. The 
authors conclude that, “[t]he good news is that we 
believe that the problem can be analyzed not by 
educators inventing a new vocabulary for finance and 
planning (as some have effectively tried to do over 
the years), not by their denying the need for such 
tools (as others have often tried), but by using the 
tools that are (slowly) being used in universities to 
solve more general and management problems” 
(Bacsich et al, 1999, p. iii). 

Various sectors or types of institutions in Great 
Britain were surveyed for the Costs of Networked 
Learning project and the study’s conclusions are 
important to consider at the outset of this Handbook. 
First, no one really knows how much networked 
learning actually costs, primarily because little 
literature in this area exists. Bacsich, et al describes 
the number of publications as “confmable, with a 
slow rate of accretion. The sources are diverse, with 
only a small proportion of direct relevance and high 
quality” (1999, p. 75). There are relatively few 
studies that examine costs and cost-effectiveness in 
computing, Bakia (2000) also finds. The author 
outlines the methodology of a World Bank study that 
looks at the cost of information and communication 
technologies at selected institutions in developing 
countries. Second, the topic of costing technology- 
enriched courses is very nebulous. Ash and Bacsich 
liken the cost inquiry to “weighing air,” a phrase 
which “describes the process of quantifying 
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something that quite definitely exists but is normally 
invisible and can only be measured by using special 
tools” (Ash and Bacsich, 1999, p. 2). 

Rumble (1986, 1992, 1997, 2002) has also 

contributed significantly to the study of technology 
related costs in developing countries. In his most 
recent effort, “Analyzing Costs/Benefits for Distance 
Learning Programs,” he explains that “each 
technology used also has a different cost structure” 
(Rumble, 2002, p. 62). 

Finally, the work of Anthony Bates (e.g., 1995) in 
Canada has been invaluable for educators who need 
to do practical studies that look at both program 
performance and costs. 

The United States: The cost and profitability of 
online education are analyzed in a series of studies 
commissioned by the Alfred P. Sloan Foundation. Six 
institutions that received grants from Sloan’s 
Asynchronous Learning Network to develop online 
programs were used for case studies. The studies vary 
considerably in what they count in costs and 
expenses (Carr, 2000). These case studies and other 
materials have been incorporated into myriad 
conference presentations and publications, among 
them the three published volumes of Online 
Learning . Among the questions raised in Volume 3 
is: “How can schools drive down costs and prices to 
achieve capacity enrollment while maintaining and 
improving the quality their distinctive missions have 
established?” For more information, see 
http://www.aln.org/ . 

Beginning in 1996, the Andrew W. Mellon 
Foundation has funded twenty-five projects as part of 
its “Cost-Effective Uses of Technology in Teaching” 
program. This program is responsible in part for the 
development of the Costs Project, the WICHE 
Technology Costing Methodology (TCM) Project, 
and this TLT Flashlight Cost Analysis Handbook. 
Institutional projects have three components: (1) 
measures of pedagogic effectiveness, (2) measures of 
costs of teaching, and (3) assessment of 
cost-effectiveness, as a ratio of (1) and (2). The cost 
model methodology in each study varies, though over 
time many similarities and best practices have been 
shared and incorporated. 

One interesting variation in the emerging 
development of cost models comes from the CEUTT 
project at the University of California, Davis. The 
costing work of Maher and others focuses on two 
types of analysis - the cost of acquiring resource 
capacity versus the cost of using resource capacity. 
Developing capacity to use technology (e.g., hiring 
new faculty and support staff; wiring buildings) can 


sometimes be very expensive. Using capacity (e.g., 
allocating the time of staff to develop and teach a 
new course) requires a different type of measure and 
is not necessarily less costly (Maher et al., 2002). 

The mission of the Western Cooperative for 
Educational Telecommunication (WCET) is to 
advance the effective use of technology in higher 
education. WCET is a program of the Western 
Interstate Commission for Higher Education 
(WICHE). One of its first steps in dealing with cost 
issues was to help create The Flashlight Project in 
1992 and begin the work on cost studies that led to 
this Handbook. In 1997, WCET partnered with 
Dennis Jones of the National Center for Higher 
Education Management Systems (NCHEMS) to work 
on standards for cost measurement. Out of this 
partnership came the Technology Costing 
Methodology (TCM) project, funded through a two- 
year project by the Fund for the Improvement of 
Postsecondary Education (FIPSE). Information about 
the TCM projects, including case studies and a “TCM 
Tabulator,” is available at 

http://www.wiche.edu/telecom/proiects/tcm . 

Over time, TCM has evolved into a complex 
methodology with its own instructions, training, 
tools, and case studies. The basis of the methodology 
was NCHEMS’ 1974 “Cost Funding Principles.” 
These principles are at the heart of a variety of 
models of the cost of instruction that have been 
created over the years. In collaboration with others 
working in this field, these principles were refined 
through new case studies involving technology 
supported instructional programs. 

Seventeen institutions participated in pilot testing the 
TCM project. Some of these studies found that their 
technology-mediated delivery systems cost more than 
campus-based equivalents and that differences in 
costs arose for different reasons, depending on the 
method of delivery. A TCM 2 Project is underway, 
designed to standardize the WICHE methodology 
and further assist with policy decisions about 
technology. Flashlight is contributing to TCM 2 as 
well. 

The TCM focuses on direct costs that may be 
allocated back to instruction. As in the work of Jenny 
(1996) by NACUBO, the course is the unit of 
analysis. Nine steps are used in documenting the 
costs of a course, among them establishing an activity 
structure, identifying resources, assigning costs to 
activities, and documenting student headcount and 
credit hour activity. Costs are calculated per student 
and per credit hour. There is also a focus on 
underutilized capacity, similar to that discussed by 
Maher (2002). 
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Frank Jewett, drawing on what he had learned from 
an earlier Mellon- funded project, helped WCET 
shape the emerging TCM through his work on 
campus level costs, in combination with the TCM’s 
course level costs. While with the California State 
University system, Jewett developed the “Bridge 
Model” to compare the cost of expanded instruction 
with mediated versus classroom delivery. When the 
Mellon- funded project ended, Cal State allowed 
WCET to take over the project. It was recognized 
that the “full-blown BRIDGE project was beyond the 
needs of most of the pilot sites, so a new simplified 
tool, mini-BRIDGE, was created to assist in making 
the link between the two projects” (Jewett, 2002, 
p. 5). Jewett applied the mini-BRIDGE model at each 
TCM pilot institution. The recently published results , 
termed the “TCM/B RIDGE Project” (Jewett, 2002), 
include the methodology, cost calculations and 
comparisons, outcomes, and case studies. The report 
allows for some level of comparison of costs, though 
it is recognized that the “calculation of costs based 
upon the TCM Handbook or any other methodology 
is a very different thing from the comparison of 
costs” (p. 14). 

The Pew Grant Program takes a different approach to 
cost-efficiency in Course Redesign conducted by the 
Center for Academic Transformation at Rensselaer 
Polytechnic Institute. The purpose of the program is 
to “encourage college and universities to redesign 
their instructional approaches using technology to 
achieve cost savings as well as quality 
enhancements.” More information is available at 
http://www.center.rpi.edu/PewGrant.html . The 

Course Redesign process involves using a “Course 
Planning Tool” or spreadsheet to compare before and 
after costs of implementing a redesign. There are 
many ways to redesign a large, lecture section of an 
introductory course, which is where the project 
focuses most of its attention. These include keeping 
enrollments the same, but reducing instructional 
resources; increasing enrollment but not resources; 
and reducing the number of course repetitions 
required to pass a class. Instead of the traditional 
focus on contact hours, there is greater reliance on 
asynchronous, self-paced learning. Faculty time-on- 
task is shifting to technology, which requires 
challenging existing assumptions that “get in the 
way” (Twigg, 2002). 

Some of these approaches incorporate traditional 
accounting and cost analysis procedures, while others 
allow for hybrid models. The CEUTT project lets 
institutions choose their own methods for costing and 
quality assessment, within a range of efforts 
supported in the literature. The projects build on the 
success of earlier grants and have developed a 


consistency in method over time. Many of the 
CEUTT projects use Activity Based Costing (ABC), 
defined later in this section and built into most of the 
Flashlight case studies. 

What is important to recognize is that, unlike the new 
NACUBO methodology for calculating the cost of 
undergraduate education, there is no one right way to 
calculate the cost of using technology. There are 
many different ways to get at different questions. If 
your question matches that used by TCM or Bridge 
or Pew, then it is worthwhile to explore these 
approaches. However, this Handbook goes far 
beyond a single method, giving you the knowledge 
and tools to build cost models unique to your 
institution and that will provide you with important 
feedback about the issues that matter most to you. 

How does this Handbook relate to 

OTHER EFFORTS IN MODELING COSTS? 

Many of the other programs discussed (e.g., Pew, 
TCM, BRIDGE) begin with a common purpose: to 
compare two or more programs, or types of 
programs, each of which uses technology in a 
particular way to educate students. The question is: 
which is cheaper, while also understanding the 
impact on quality. In order to use the models, users 
must study the exact same question, adhering to the 
model’s definitions and standards. The benefit of this 
approach is the ability to compare costs across 
programs, departments, and institutions. 

The problem with this approach is that most of the 
reasons people have for studying costs do not fit 
easily into one of these preestablished models. 
Instead, this Handbook teaches how to apply a 
flexible set of methods to do studies tailored to your 
own problems and information resources. The 
particular method emphasized in this Handbook, 
activity-based costing, is also well suited to develop 
studies that can show how to change local practices 
in order to improve results while controlling costs. 
Activity-based costing is also a good fit with 
Flashlight’s activity-based methods for studying 
program performance and outcomes. 

It can be quite valuable to study the models discussed 
in earlier sections so that you may draw from them or 
participate in them if appropriate. We suggest, 
however, that you use them as a starting point. 
Explore the case studies of this Handbook as a way to 
get started; then focus on building your unique model 
to meet the questions that face you and your 
institution. 

Cost analysis of a teaching innovation ought to be an 
inseparable part of a larger inquiry. Such an inquiry 
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should typically seek answers to at least three types 
of questions: 

(1) What people are actually doing in a program 
(e.g., how they are actually using technology 
to support teaching and learning activities) 
and why they are choosing to do things that 
particular way. The latter type of 
information can be very helpful in 
improving activities; 

(2) The outcomes of those activities; 

(3) How do those activities use resources; what 
kinds of stresses do those activities create on 
budgets and staff? How might those 
stresses be reduced? 

It is with this third set of questions that concern this 
Handbook. 

Outline of the Handbook 

Readers may want to follow the Handbook 
systematically when looking at a particular question 
of interest. The general examples in each section of 
Part II include basic spreadsheet models to collect 
data and build ratios and equations that relate 
different variables and cost drivers. Part III includes 
the worksheets for a sample application of the model. 
Its intent is to breathe life into the process by placing 
it for a specific question - in this case comparing the 
costs of a traditional section of a course with a 
section that uses Internet technology and meets less 
often. 

Appendix A is a glossary of terms used in the 
Handbook. 

Appendix B presents in detail the seven case studies 
referenced in the body of the text. These help the 
reader see more of the methodological choices and 
data issues involved in the costing process. 

In the first case study, Joseph Lovrinic and James 
Johnson of Indiana University report on the “Cost 
Implications of Technology Use in the IUPUI 
English Department’s Core Writing Curriculum.” It 
is useful to read this case in conjunction with 
Susanmarie Harrington’s Flashlight study of 
educational results of those same courses, which 
appears in the Flashlight Evaluation Handbook; the 
Harrington study is also available on the Web at 
http://www.tltgroup.org/resources/fcasestu.html . 

David Pope and Helen Anderson analyze the costs 
and performance of different approaches to 
organizing undergraduate engineering laboratories at 
the University of Pennsylvania in the second case 
study. Their study highlights one key advantage of 
Flashlight’s activity-based approach over more 


traditional methods. Flashlight's activity based 
costing approach can separate the onerous and 
burdensome activities (e.g., faculty and TA time 
spent in managing grades and student data) from 
those activities that are fulfilling and productive (e.g., 
coaching students in the lab). Armed with a model of 
how time is spent on the onerous activities, the staff 
can then rethink the process in order to reduce time 
spent on them. Another nice feature of this chapter is 
the insight that laboratories in different disciplines 
have shown by asking students to do experiments that 
look different on the surface but have the same basic 
structure (suggesting that in the future common labs 
with common software can serve multiple 
disciplines). Why is this important? Cost analysis is 
often seen as an attempt to slice and dice academic 
life by people who do not understand the first thing 
about that life. The approach to analysis 
recommended by this volume is, we hope, the 
opposite: helping educate people so that they can 
examine their own working lives in a new way. The 
Pope-Anderson study is a good example of that kind 
of sensitive self-analysis by an academic program. 

Chris Geith and Michelle Cometa of the Rochester 
Institute of Technology share the cost analysis results 
of their comparison of distance learning and on- 
campus courses, in the third case study. Their study 
highlights several important lessons, including the 
enormous variability of how individuals can spend 
their time (variation sometimes masked by cost 
models) and the potential usefulness of separating 
novices from veterans when analyzing how much 
time it takes to teach courses at a distance. 

In the fourth case study, John Milam describes the 
cost assessment methodology used for George Mason 
University for its Andrew W. Mellon Foundation 
grant project on "Cost-Effective Uses of Technology 
in Teaching." Milam’s analysis draws on decades of 
research and experience from the NACUBO and 
NCHEMS literature to provide a hybrid approach to 
the pure activity-based analysis described in this 
Handbook. It is a particularly useful approach if the 
policy options being studied deal with staffing and 
space (as opposed to the reorganization of teaching- 
learning practice) and/or if it is not possible to gather 
faculty workload data on time spent on various 
activities and tasks. Milam’s chapter is also unusual 
in that it provides a model for revenue implications of 
the programs he compares. 

Many educators assume that cost analyses only need 
to be done once. In the fifth case study, Susan Tucker 
and Jamie Kirkley analyze costs in a complex, multi- 
institution project that helps teachers learn to use 
technology by involving them in the development of 
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online instructional materials that teachers can use. 
Their study demonstrates how an ongoing process of 
cost modeling can become part of the life and 
evolution of a program, guiding change in its 
strategies and structures. 

The sixth case study reports how Tom Henderson and 
Gary Brown of Washington State University compare 
three different materials development efforts 
supported by their University. One of the most 
interesting lessons from their study is how projects 
that apparently cost the same can look very different 
when one compares the cost per hour of student use 
and benefit from the materials. 

In the final case study, Craig Blurton, Anita Lee, and 
Winston Ng compare two methods of making 
computing widely available at the University of Hong 
Kong: a laptop-leasing program versus equipping a 
large number of computer laboratories for students. 
This study provides vivid detail about the practical 
difficulties of doing such a study and how one team 
dealt with those problems. Blurton, Lee and Ng draw 
parallels between their quest for cost data and Ahab’s 
pursuit of the White Whale in Moby Dick. 

The variety of these cases demonstrates the flexibility 
of the Flashlight approach to cost analysis. The ideas 
described in this Handbook can be applied to a 
surprisingly wide range of issues involving the use of 
time, money, space and other resources. 

Worksheets. Each of the worksheets presented in 
Parts II and III is available in electronic format. A 
diskette containing these spreadsheets in Excel 97 
and Lotus format is included with this Handbook. 
Two files are listed in each file format, for example 
part2.xls and part3.xls. 

Who should do your study? 

Teams, not isolated individuals, ordinarily create cost 
models. That is because few individuals have the 
right mix of expertise in content, modeling, analysis, 
and dissemination of the results. For most such 
studies, a substantial amount of work is required— 
another reason for using a team. 

The team should include expertise in the program or 
activity being studied (often provided by people who 
work in the program), in the arts of model building 
(business office staff or other management experts), 
and in the ways your institution stores data that might 
be relevant for creating the model (institutional 
research or business office staff). 

There is an obvious tension between building a team 
big enough to include the needed expertise and 
representation and building a team small enough to 


work efficiently. One way to resolve this tension is to 
have a small “operating team” and a larger “steering 
group ”. The Steering Group could include 
representatives of the offices or types of people by 
whom whose findings will influence the most, such 
as the provost’s office. This group should include 
students if you hope to influence the ways that 
students use resources or if you need students to 
answer your survey or interview questions. 

Where can you find team members? If your 
institution has a Teaching, Learning, and Technology 
Roundtable, it should provide an excellent source of 
guidance about who should be involved in such a 
study, how it might be funded, and how it should be 
focused. An assessment office, institutional research 
office, or teaching-learning center may already be 
involved in efforts to use Flashlight tools at your 
institution. This is a good place to go for help in 
formulating your initial cost questions and in 
gathering data. Academic computing or instructional 
technology offices are also involved in a variety of 
similar tasks, though with less of an assessment or 
reporting mindset. 

Hint: Teams like this sometimes fall apart but, if you 
know the traps, they can often be avoided. For 
example, members may initially overestimate one 
another's abilities to contribute. A business officer 
may assume that faculty members have a precise 
record of how they spend their own time and know 
everything that students do. A faculty member may 
assume that a business officer knows how the 
institution spends every cent of its money, knows the 
cost per square foot of space, and is a complete 
master of accounting minutiae. If people see that 
their expectations of one another have been 
unrealistic, they sometimes overreact. Faculty may 
leap to the conclusion that the financial experts 
understand nothing of what is really involved in 
teaching. Business officers may conclude that faculty 
believes money grows on trees. The truth may lie 
between these two extremes. 

Some advice: Be patient and keep going, value the 
lenses of different assumptions, and keep a sense of 
humor. Taking this kind of a team approach will also 
promote critical buy-in and ownership of the project 
from the people most affected by it. 
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PART I: THE FUNDAMENTALS 


It is difficult to figure out how much higher education 
does cost. Prices and accounting methods vary by 
type of institution and with the economic situation. 
Services may be inexpensive at some institutions 
because of low resource costs or hidden subsidies. 
Yet similar services may be quite costly at others, 
also for local reasons. Complicating the question 
further is the rapid and not always predictable change 
in technology prices and performance. Cost 
considerations are difficult to express and change so 
rapidly that people need to be careful about making 
generalizations not supported by empirical data. 
Caution flags should go up whenever you hear 
someone say, “The nation can teach English 
composition more cheaply if it uses technology X.” 
The issues are more complicated and the Flashlight 
Program’s tools and training are designed to assist 
you in coming to terms with them. 

An important distinction needs to be made between 
research on costs, which produces generalizations 
that are true in the average for many schools, and the 
evaluation of costs, which produces results that are 
true at a particular school at a point in time. The 
methodologies described herein may be used for 
either, but the purpose of the Handbook is evaluation 
on a local scale. It is quite difficult to do 
inter-institutional comparisons of program costs and 
this Handbook does not deal with that particular 
challenge. 

So what are sensible activities on which to focus and 
what are good questions to ask that might reveal how 
resources could be stretched and used in more 
productive or fulfilling ways? 

Some of the toughest questions in the study of 
teaching, learning, and technology revolve around the 
use of resources. The first challenge is to get past the 
idea that there is some universal, normal cost for 
using a particular technology in a particular way. In 
his classic study of “ The Costs of Higher Education: 
How Much Do Colleges and Universities Spend per 
Student and How Much Should They Spend ?”, 
Howard R. Bowen (1980) explains that there is no 
universal standard for how much money or time is 
enough in higher education. Bowen’s findings 
suggest that cost savings depend more upon how 
technology is used than on what technology is used. 
For example, two institutions might invest the same 
amount in Internet infrastructure. One of them might 
make no significant changes in the academic 
program, so the cost of the technology would simply 
be added to other costs. But the other might use that 


infrastructure to reorganize the academic program 
and library services, incurring additional up-front 
costs but perhaps cutting operating costs. 

Once you have decided to study, what resources are 
required to carry on educational programs or 
processes that rely on, or support, technology use? 
What standard of judgment should be used to decide 
whether an activity has cost “too much” or has 
“saved money” (and compared with what)? Which 
expenditures should be included in cost models? 
What other resource uses should be included, or 
excluded? Space? The value of student time? 
Network services? Even if we know the history of 
how we used resources yesterday, does that tell us 
anything about how the same people and activities 
will use resources tomorrow? 

Despite the difficulty of such questions, it is still 
possible to make some assumptions, model how 
activities consume time and money, and then make 
suggestions for how to make more productive, 
comfortable use of available resources. For example: 

• As describe in the University of Pennsylvania 
case in Appendix B, David Pope and Helen 
Anderson studied ways of improving 
undergraduate engineering laboratories while 
reducing costs. Their successful strategy helped 
Penn focus on productive use of faculty time 
(coaching students) while cutting more 
burdensome uses of time (grading and record 
keeping), cutting equipment costs, and getting 
more productive use of the laboratory space. 

• In another appended case, the University of 
Hong Kong compared their costs for two 
different strategies of making computers 
available to students such as leasing laptops 
versus providing the same computer capacity 
through publicly equipped labs. For them, the 
leasing program saved money. 

• Washington State University analyzed the costs 
of three projects, each representing a different 
approach to improving courses with technology. 
Although the three projects had cost about the 
same amount of cash, Tom Henderson and Gary 
Brown found that one of the three strategies was 
far more costly than the other two when they 
asked how many hours of student benefit were 
created by each dollar of investment. 

• Sometimes student outcomes data are critical. In 
a study not included in this volume, Baruch 
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College studied related initial investment to the 
number of students passing the resulting course. 
Baruch College had started an experimental 
course called “College Literacy” designed for 
students with poor skills in reading and writing. 
A section that did not use computers suffered a 
failure rate of almost 50%. Students did better in 
an experimental course that featured 
considerable use of a real-time computer 
conferencing system called Daedalus. The 
instructor's assessment of the exit exams, each of 
which was graded by three independent readers, 
was that "the students in the computer-enhanced 
section consistently wrote longer ...essays rich in 
ideas and details, organizationally complex, 
remarkable in their fluency." Pass rates for the 
experimental course were also much better 
(about 75%). Although most costs were the same 
(space, faculty time) for the two courses, the 
conferencing software added about 7% to the 
cost per student. However, so many more 
students passed this course that Baruch estimated 
that the costs per passing student were 29% less 
than the costs per student passing the course that 
did not use computers. Cost per passing student 
can be a good way of expressing the costs of 
activities meant to improve retention. 

Creating models of resource use is not easy and is 
always controversial. However, a properly designed 
study can uncover facts that point toward real 
improvements in both education and costs. 

Details: What is an “Economic 
Model”? 

An economic model is a summary of how people 
routinely use resources in order to accomplish 
something. Such a model is made up of four major 
components: (1) activities (what is done); (2) 
organizational units involved in carrying out those 
activities; (3) inputs (resources needed to carry out 
the activities); and (4) outputs (results of those 
activities). The economic models described in this 
handbook are expressed as numbers and relationships 
in a spreadsheet; there will be more to say on the 
possibilities and limits of such models later. 

Defining “output”: In this Handbook, the term 
“output” refers to a completed activity while the term 
“outcome ” refers to benefits or problems associated 
with that output. 

• Example of output: a course enrolling 30 
students has successfully been offered and 
completed. 


• Examples of outcomes: most graduates found 
jobs; most students completing the course were 
better human beings than when they started; 
minority retention was unusually good. 

Another ambiguity of terms is that, depending on 
what you are analyzing, something that is an output 
in one model may be an input in another model. For 
example, if you are analyzing the resources needed to 
provide network services, student Internet accounts 
may be an output. In contrast, for a model of online 
instruction, student Internet accounts might be an 
input. 

Let us define each term in a bit more detail. 

Activities and tasks. Activities are processes or 
functions performed by individuals and 
organizational units. A study topic usually can be 
broken down into a number of different activities, 
each of which may be made possible by work or 
services from more than one organizational unit. For 
example, in creating a course, one instructional 
activity might be creating a CD-ROM of 
supplemental course materials. 

Each activity can be further broken down into tasks. 
For example, in producing a CD-ROM, tasks include 
outlining content based on the course syllabus, then 
locating, evaluating, obtaining copyright permission, 
and converting content into electronic format. It is 
possible to break an activity down into an infinite 
number of tasks, but it is not a good idea. Activities 
and tasks need to be recognizable ways of describing 
action and its resource requirements. If a total process 
is broken into too few activities or tasks, one gains 
too little insight into how it uses resources. Yet too 
many tiny categories create a huge amount of data 
collection work for the study team. 

(Organizational) units. Academic and administrative 
units perform the activities, with help from various 
support units. Developing the CD-ROM may evolve 
with some of activity taking place as part of the 
academic department’s budget, some from the 
financial aid department’s budget (supporting student 
assistants), some from the technology support 
operation’s budget, and so on. 

Inputs. To carry out the activities, the units require 
such resources as people’s time, buildings, 
equipment, services (e.g., computer networks), and 
money for expenses. Models usually need to find a 
common means of expressing the value or quantity of 
such resources; money is the most frequently used 
measure. 

Outputs. In our example, the output is a completed 
CD-ROM. There are two ways to think about outputs 
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- generally and in terms of performance measures or 
metrics. For some decisions, it may be important 
simply to know the total cost of producing the 
CD-ROM (including various ordinarily hidden costs). 
However, much more may be learned by looking at 
different metrics for expressing the output. What is 
the cost per user of producing the CD-ROM? If using 
the CD is intended to improve the retention of 
students in a course, what is the cost per retained 
student? 

As you can see, there are many ways to express 
outputs. The general output is really a function of the 
resource question of interest, defined by what it is 
you want to study. The specific output is a 
performance measure for examining how much the 
output cost, using a specific metric or indicator. 

Relating the inputs to the outputs: 
Activity-Based Costing 

Once you identify the inputs, units, activities, and 
outputs you are interested in evaluating, examine the 
quantitative relationships between them using an 
approach called activity-based costing. How many 
dollars, how much space, how much time and what 
other resources were required to teach that class, to 
carry out that style of assessing student learning or to 
develop such a CD-ROM? 

Differences between traditional accounting and 
activity-based cost models. Traditional accounting 
breaks down costs and revenues by organizational 
unit (even though each unit is carrying out, or 
contributing to, many activities). In contrast, 
activity-based costing breaks down costs by activities 
(even though each activity is the shared responsibility 
of many organizational units). These costs may cut 
across organizational boundaries and include all 
relevant resources (e.g., use of time, space), even if 
no unit is currently carrying that ‘cost’ as part of its 
operating budget for the year. Traditional accounting 
covers the whole organization and is rather simple in 
at least one respect: each person is usually the 
budgetary responsibility of one unit. In contrast, 
activity-based cost models focus on one process 
composed of several activities. 

Activity-based cost models have at least three 
advantages over traditional accounting: 

a) They can express costs in a way that can be more 
easily related to benefits, 

b) Activity-based models are usually designed to 
help people make specific types of decisions 
(should we teach this way or that way, lease 
computers or equip laboratories, or reconfigure 


our undergraduate engineering laboratories?), 
and 

c) Activity-based models make it possible to reduce 
uses of time or other resources that are 
unpleasant or unproductive, while investing 
more time, space or money in ways that are 
fulfilling or successful. For example, in the Penn 
study of undergraduate engineering laboratories, 
the study helped the University take steps to 
reduce faculty and TA time spent on record 
keeping while enhancing the time spent on 
coaching students in the laboratory. 

Cost drivers. A cost driver is an element of a process 
that, if changed, would have a significant impact on 
the resources needed. 

For example, a team might “tweak” a model to see 
how a change affects total resources required by 

a) changing the number of students to be handled, 

b) changing the assumption of how many students a 
faculty member can teach, or 

c) changing the assumption about how much study 
materials would cost per handout (e.g., if 
handouts were made available on the Web and 
sometimes printed by students instead of being 
mass-printed and sold). 


Underlying assumptions 

In constructing any model of resource use, certain 
assumptions usually need to be made. For example, it 
is assumed that: 

❖ Modeling processes take place more or less the 
same way each time they occur. Notice that 
phrase “more or less”, we know that people 
never do a task the same way twice. The 
question is whether their patterns of using 
resources are consistent enough to make it worth 
learning from experience. It is not always easy to 
determine whether there is enough continuity 
linking past, present, and future. People can be 
relatively isolated from one another in higher 
education so that one may assume everyone is 
doing pretty much the same thing, and be wrong. 
Alternatively, one might be fooled by radical 
differences between two people and not realize 
that the mass of faculty are, on the average, 
doing things in pretty much the same way. 
Another mask that can be deceptive comes from 
technology. Many people assume that, when 
technologies change, all past experiences 
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become irrelevant. This purposeful amnesia may 
prevent them from noticing that patterns of waste 
and failure repeat themselves with each new 
generation of investment (e.g., Ehrmann, 2002). 

❖ Costs can be summed with appropriate metrics. 
For example, your model may implicitly assume 
that, if it costs a certain amount to create 40 
copies of a book, it will cost about twice that 
much to create 80 copies. This kind of 
assumption may be good enough for your 
decisions, especially if, when documenting your 
model, you note the range of operation within 
which its assumptions are valid. You might, for 
example, write in your report that “we assumed 
in creating this model that no fewer than 20 and 
no more than 100 copies would be made at any 
one time. Below 20 copies, the price per copy is 
much higher, while above 100 a different and 
much cheaper production method would be 
used.” 

❖ Accurate workload data can be gathered. When 
asked about how they spend their time, faculty 
members, staff, and students can and will answer 
truthfully if they do not feel threatened by the 
study and if they have some interest in its 
accuracy. If it is not possible to gather data about 
how people spend time on specific activities, it is 
sometimes possible to use alternative methods 
outlined in the appended report about work at 
George Mason University (see appendix B). 

♦> One dollar (or hour of work) is like another. 
This assumption is often made in cost models but 
we suggest that it be re-examined. It is quite 
possible to designate specific uses of time as 
negative (to be reduced if possible) and others as 
positive (to be increased if possible because they 
are fulfilling for the people involved and 
effective for the institution and its students). In 
fact, the ability to look differently at different 
uses of time is one of the potential strengths of 
activity-based cost models. 

What Activity-Based Costing 
is NOT 

Activity-based models do not replace accounting or 
the traditional method of indirect cost recovery 
required in sponsored research administration. They 
have a different purpose: to evaluate a program or 
process in order to make decisions about it, usually in 
conjunction with other information about the 
program such as the quality of its outcomes, the need 
for its outcomes, barriers to the program, and so on. 


Activity-based models are designed to answer quite 
specific questions - so be careful about applying a 
model in an inappropriate context. Suppose, for 
example, that you are interested in the student 
projects in studio classrooms that might improve 
retention on campus. You gather data and build a 
model to help you estimate those savings. Later, you 
become interested in another proposal to improve 
retention that (as it happens) involves relying on the 
Web instead of physical classrooms. Your original 
model might help you answer that question too, but 
there might be problems. For example, in building 
the original model of several classroom-based 
options, you may have decided to ignore the cost of 
space because all the options you were considering 
used the same amount of classroom space. Unless 
you modify the model to include the cost of 
classroom space, it will make the Web alternative 
seem comparatively more expensive than it is in 
reality. Model making, like other forms of evaluation, 
requires a consistent, clear focus on the resource 
question of interest. 

Third, activity-based costing, like other forms of 
evaluative study, does not dictate what you should do 
next. Evaluative studies are necessarily about what 
has happened or about people’s assumptions about 
what might happen. Decisions require a creative leap 
and a value judgment as well as data. Stretching 
resources is often a good thing, but not always. Thus, 
findings should shape your mind, not determine it. In 
an example described in Appendix B, Washington 
State University (WSU) compared several methods 
of developing technology-based course materials. 
The most expensive initiative involved the 
development of a computer-based tutorial that could 
be used for a relatively short time by a small number 
of students, when cost-effectiveness was calculated in 
terms of dollars spent per hour of student use. What if 
the modules are essential to recording and preserving 
Native American languages for future generations? In 
that case, the institution might reasonably decide to 
produce more such modules, even though they are 
expensive, if conserving cultural diversity is valued 
as part of the institutional mission. The lesson is that 
tailored modules usually need to have very specific 
and justifiable goals to justify their relatively steep 
costs per hour of student use. 

A fourth cautionary note about activity-based costing 
is that most models do not directly relate cost to 
quality, though occasionally it can be done. You will 
usually need to use other means, such as the 
course-specific assessments of achievement and 
evaluation tool kits developed by the Flashlight 
Program, to study questions related to quality. 
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PART II: BUILDING YOUR ECONOMIC MODEL 


You can complete the Economic Model in seven 
basic steps: 

1. Identify resource concerns and the specific 
questions to answer. 

2 . Identify your outputs. 

3. Identify the activities required to produce your 
outputs. 

4. Identify the academic and support units that 
participate in these activities. 

5. Identify the resources these units consume in 
their activities. 

6. Calculate costs for these activities. 

7. Tally the costs of all activities to arrive at your 
output costs. 

Step 1: Identify resource concerns 

AND THE SPECIFIC QUESTIONS TO ANSWER 

Evaluative studies always begin with a comparatively 
ill-defined sense that data needs to be gathered. The 
first and often hardest step is to define the object of 
the study. This often requires considerable time and 
debate. The seven teams whose work is reported in 
this Handbook and its appendices drew quite 
different conclusions about what to study. 

❖ At Indiana University Purdue University 
Indianapolis (IUPUI), team members decided to 
analyze sections of composition courses that 
were offered in computer-equipped classrooms, 
comparing them with other sections of the same 
courses offered in traditional classrooms. Using 
the Flashlight Current Student Inventory, the 
team was also interested in whether the sections 
in the computer classrooms were taught 
differently and better than sections taught in 
traditional classrooms. 1 

❖ At the University of Pennsylvania, a new way of 
organizing undergraduate engineering 
laboratories was analyzed. 

❖ Team members at the Rochester Institute of 
Technology (RIT) had several resource concerns. 
They wanted to: (1) identify all costs associated 

1 This study by Susanmarie Harrington was reported 
in the Flashlight Evaluation Handbook and Current 
Student Inventory and is also available on the 
Flashlight Web site at 

http://www.tltgroup.org/resources/fcasestu.html. 


with teaching a variety of courses using distance 
learning technologies, as well as all costs 
associated with teaching the same courses on 
campus; (2) discover the revenue benefits of 
distance learning courses; and (3) determine 
costs for distance learning staff time and services 
in order to develop an internal price sheet. 

❖ Tucker and Kirkley wove cost analysis into the 
life of a three-year grant funded project to 
develop online materials and educate teachers; 
cost data helped shape the evolution of project 
strategy. 

❖ At Washington State University (WSU), users of 
the model compared the cost of several 
alternative ways of spending money on using 
technology to support teaching and learning. 

❖ In conducting its cost analysis of seven 
innovative courses and their traditional 
classroom counterparts, George Mason 
University (GMU) compared the cost- 
effectiveness of different combinations of 
technology and personal contact between 
instructor and student. 

❖ The University of Hong Kong focused on 
infrastructure: whether to expand a pilot program 
of leasing laptops to students or focus on making 
computers available through public labs. 

The variety of these topics helps illustrate the need 
for this Handbook. Other cost efforts either simply 
publish results (X is usually cheaper than Y) or 
distribute models that are useful only for one class of 
cost problem (is one type of course cheaper or more 
expensive than another type). In contrast, this 
Handbook will help you focus your attention 
wherever it is most needed. 

A Fictitious Example 

For the puiposes of illustrating the description of the 
Flashlight approach to activity-based costing in this 
Handbook , we will follow the development of a 
model of a fictitious example: the cost implications 
of offering a non-traditional class section of a popular 
course, Sociology 101. In step 1, our team decides to 
ask the following question: 

What is the difference in costs between offering a 
traditional section of Sociology 101, which meets 
twice a week for fifteen weeks, and another section 
that meets only a few times and does most of its work 
on the Internet? 
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Hints for asking good questions 

First, focus on a few activities or processes. These activities 
should he: 

♦> technology-related (the activity depends on 
technology, your use of technology depends on the 
act i\ itv, or both); 

♦> educationally crucial (the activity has a pivotal impact 
on student outcomes); 

♦> resource-intensive (the activities seem to involve a lot 
of money, people's time, network use, space, and 
other limited resources); 

♦♦♦ problematic (more information could give you the 
opportunity to change the activity in beneficial ways); 

♦♦♦ promising, arousing hope that the cost of the study 
will not exceed the potential gains from the study! 
Among the gains to consider: stretching or 

reallocating resources (including uses of time that are 
most satisfying and productive), gaining new 
resources, defending budget allocations. 

Cautionary note: most cost studies are themselves costly, 
not just for the team but even more, for the people who 
need to prov ide data. Collecting data from staff or students 
about how they spend their time is quite time-consuming 
(and therefore costly), tor example. A useless study, 
unfortunately, can be just as expensive as a study that is 
worth ten times the cost of doing the study. 

To help avoid doing a Hawed study that is looking at the 
wrong topic, start by creating a Fictional version of your 
completed study. In other words, as soon as you have 
picked your topic, make up the cost numbers and present 
the results to the decision-makers represented in your 
steering group. Then ask the steering group members how 
they would use the findings. Make up estimates of what it 
cost to collect that data (including the time spent by people 
contributing data; you might imagine that you need to pay 
them for their time and add that inv oice to the eost of doing 
the study). If your steering group replies that “this data 
doesn't give us enough ev idence for action," or “the study 
would have eost us more money than it saved," you need to 
go back to the drawing board. 


Step 2: Identify your outputs 

When you identify the resource question you are 
most interested in, the general outputs are often 
outlined for you. If the process to be studied is the 
creation of CD-ROMs, then CD-ROMs are usually 
the output of interest. If you are comparing different 
ways of developing and supporting online courses, 
then the number of courses actually offered (or the 
number of students educated) will probably be the 
most important output. 

For the purposes of this example, we are interested in 
comparing two classes. Both are sections of the same 
course, Sociology 101. The same fifteen weeks of 
course content is covered in each section, guided by a 
syllabus on file with the department chair. The 
catalog listing does not differentiate between the two 
sections, other than to say that one is online and 
meets less frequently on campus. 

Comparing outputs. After you determine general 
outputs, you will want to set specific performance 
measures and to think about possible metrics that 
may affect the variables in the model. In defining 
specific outputs that may be expressed as 
performance measures, it is helpful to begin by 
looking at similarities and differences between the 
two course sections. 

The traditional course section: 

❖ requires developing a syllabus and printed 
curricular materials; 

❖ is taught in a conventional classroom, meeting 
twice a week for fifteen weeks; 

♦> serves 35 students who are freshmen or 
sophomores; 

❖ requires an instructor at each class meeting. 
She/he also does routine course administration 
tasks such as monitoring enrollment, grading, 
and keeping regular office hours; 

❖ uses a textbook and a chalkboard format to relay 
information; 

❖ uses paper and pencil testing administered with 
quizzes and mid-term and final exams. 
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The technology-assisted course section: 

❖ has the same syllabus and course sequence, but 
uses electronic curricular materials 

❖ meets on campus only three times during the 
semester, for the: (1) course introduction; (2) 
mid-term exam; and (3) final exam; 

❖ serves 100 students in a mix of student levels; 

❖ requires that the instructor be physically present 
only for the three meetings. The rest of the time 
she/he monitors online discussions, responds to 
student email and phone calls instead of keeping 
office hours, updates electronic content, and does 
other routine course administration tasks; 

❖ uses the textbook for assigned readings that are 
not available online; 

❖ relies on Internet tools to provide supplemental 
information. The class website includes an 
electronic syllabus, links, online class notes, 
interactive quizzes, and graphics. A listserv and 
email are also used. 


Determine w hat outputs you are going 
to study 

As you begin work on your Economic Model, it is 
important to carefully select the outputs you are going to 
study. 

At Indiana University Purdue Ini \ ersi ty Indianapolis 
(II PI I). team members wanted to compare the costs of 
offering traditional \crsus technology-assisted courses. 
They had a difficult time making accurate comparisons 
because there were so many \ariables among the courses 
they selected to study, faculty teaching the courses had 
different levels of experience and had different motiv ations 
for teaching. Some faculty taught full time, while others 
were part time. Enrollments in the courses were affected bv 
the si/e of the classrooms in which they were scheduled. 

The University of Pennsylvania (Penn) team had a simpler 
time of defining outputs because their inquiry w as defined 
by their Mellon grant: compare the new method of teaching 
undergraduate engineering laboratories with the old. One 
important insight: pay attention to which uses of instructor 
time were seen as productive and fulfilling, and which 
other uses faculty and teaching assistants wanted to 
minimi/e if possible. In effect, these uses of time were also 
treated as outputs. 

Rochester Institute of Technology (RIT) investigators 
decided to keep their model simple in order to reduce 
variability and focus on course delivery, as opposed to 
content development. They studied eight courses and 
targeted faculty members who were experienced in 
teaching technology-assisted courses. In addition, the 
chosen faculty presented their courses in both distance 
learning and an on-campus format during the quarter the 
study was conducted. Narrowing their focus, says the RIT 
team, made it easier to draw comparisons and to design a 
model the team could use long term. 

At George Mason University (GMU). it was clear that there 
are many wavs in which technology is already being 
incorporated in teaching and learning. By designing a 
two-dimensional grid that describes combinations of 
technology ("tech") and faculty contact ("touch") in 
courses, the study suggested a new conceptual map for 
understanding educational strategies and their costs. 

The "L .earning to Teach with Technology Studio" (LTTS) 
project presents a more elusive challenge. As this ease 
study details, the outputs of their project (and their model) 
were changing every year, influencing one another. 
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Users of the model at Washington State University (WSU) 
analyzed three typical approaehes to development of 
material tor online eourses. Because they were only 
studying one example of eaeh approach, the WSU’ team 
found that eaeh output they studied "may have had more 
partieulars than generalities." making it cl i t lien 1 1 to 
generalize. Nonetheless, the results were striking and the 
methods they developed are easy to apply to new eases. 

The University of Hong Kong (IIKU) compared two 
different ways of making computers available, leased 
laptops versus publicly available machines. This raised two 
challenges of output definition. The leased laptops were 
available only to their undergraduate 'owners' but the 
publicly available labs were used by graduate students, too. 
For reasons discussed in that chapter. 1 1 K l treated the labs 
as though used only by undergraduates. The team also had 
to decide what output unit to use; they decided to use hours 
of access to machines, rather than attempt to measure 
"hours of use of machines." Their reasons were pragmatic: 
it would have been too difficult to measure hours of actual 
use. especially of leased laptops. 


Define performance measures as specific outputs. For 
our example comparing two sociology course 
sections, how might you quantify the general 
outputs? Costs per student? Faculty hours spent per 
student? 

A performance measure provides insight into the 
efficiency of an activity. There are literally thousands 
of possible performance measures that you might use. 
In picking one for your model, make sure that it will 
get at the resource question of interest, telling you 
something informative about the costs of what you 
are evaluating. 

Choosing the wrong cost measure can be quite 
misleading. For example, one of our authors, Gary 
Brown of Washington State University, has remarked 
that average cost per student credit hour should 
usually be avoided in studies of technology use (even 
though it is commonly used in other aspects of the 
cost literature). That is because specific uses of 
technology often occupy differing amounts of time. 
Brown’s suggestion can be illustrated with this 
simple example. Imagine that two courseware 
modules, A and B, are developed at a cost of $900 
each. One 3 credit course with 10 students uses 
module A for one week while another 3 credit course 
with 10 students uses module B for 15 weeks. The 
cost per student credit hour for each course is $30. 
However, the cost per student week of use for course 
A is $90 while the cost for student week of use for 
course B is $6, an order of magnitude difference! 

The results of performance indicators may be 
interpreted in different ways for different purposes. 
With an ’’accounting lens,” it is desirable to have 


more students taught per faculty, indicating that 
funds from tuition payers, donors, and taxpayers are 
being stretched to educate more students. A ’’student 
learning lens” might interpret those same results to 
mean that faculty members are overburdened and 
unable to spend enough time with students, thereby 
decreasing the chance for quality interaction. 
Thinking these issues through in advance can be 
important. For example, if both of these perspectives 
are valid concerns, the model might be designed to 
look separately at faculty time spent in direct 
interaction with students, and other Flashlight tools 
might be used to gather data directly about student 
perceptions of interaction. Perhaps some strategies 
result in both a reasonably high level of interaction 
and support, but with an acceptably low faculty cost 
per student (e.g., by enabling faculty to spend less 
time on tasks that do not directly support interaction 
and learning) 

The sociology example in this section of the 
Handbook uses the following two performance 
measures: 

Performance measures: 

1 . What is the total cost for each section? 

2. What is the total cost per weekly student course 
hour? Weekly student course hour is defined as 
the number of hours in which students are 
enrolled to sit in a course (usually 3 hours per 
week per student for fifteen weeks). 

Defining metrics for cost drivers. A cost driver is a 
factor with a strong influence on your output. 
Suppose that $300 is spent to produce the CD-ROMs 
to supplement a course. To determine the unit cost 
for each CD, you first need to know how many were 
made. The size of the production run is a cost driver. 
Since 100 CDs were made with that $300 investment, 
the unit cost is $3. If instead of making 100, you 
made 1 ,000, it is possible that the cost per unit might 
drop to $ 1 . This is one way a metric can be used to 
help ask “what if’ questions about costs. 

Step 3: Identify the activities 

REQUIRED TO PRODUCE YOUR OUTPUTS 

Your next step is to identify the activities that go into 
offering the general outputs; in our example, the 
outputs are the two courses. What activities are 
required? In our simplistic example, our two courses 
both require the same three activities: preparing the 
course, teaching the course, and grading. 

How might the activities be broken down into 
specific tasks? For the purposes of this example, we 
define a “task” as a specific job duty or responsibility 
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that an individual or an office performs. An activity 
may consist of different tasks performed by different 
offices. 


Identity your act ivi ties and tasks 

Identifying the activities and tasks needed to produce your 
output is one of the most crucial steps in building your 
Economic Model. 

Due to the nature of its resource concerns. R I I focused its 
model on course delivery and was able to build its activ ities 
and tasks around an existing course development model. 
Activities included course preparation, interaction, and 
assessment, with about ten tasks for each activity, from the 
outset, the Ri f team enlisted faculty support in identifying 
activities. It invited faculty to help develop the activity-task 
matrix, respond to surveys, and participate in mock 
interv iews that helped define the terms the team would use 
in constructing the model. 

At WSU. users of the model identified three major 
activities. Although they knew that using more activities 
would yield more information, team members worried that 
if they selected too many activities, the distinctions among 
them would blur. 

Data collection for cost assessment at (All included an 
open-ended, unstructured interview protocol to "tell the 
storv" of each class. In telling the story of their classes, 
faculty helped to document their workload, identity 
activities and break them down into tasks, and identity 
direct costs associated with the tasks. 

The University of I long Kong (IIKU) had to define the 
activities and tasks needed for setup and recurrent costs ot 
both laptop leasing and publicly available laboratories. 


As you array the list of activities for each sociology 
course, the worksheet would probably look 
something like this: 


Activity Matrix (sheet 1) 



Traditional 

Class 

Technology 

Class 

Activity 1 

✓ 

✓ 

Task A 

X 


TaskB 

X 


TaskC 


X 

TaskD 


X 

Activity 2 

* 

✓ 

Task A 

X 

X 

TaskB 


X 


The checkmarks beside Activity 1 and 2 mean that 
each activity takes place in some form in both 
classes. The X in column 1 means that Task A takes 
place in the traditional class, but the lack of an X in 
column 2 means that this task is not relevant to the 
technology class. Task A in activity 2 takes place (in 
some form and to some degree) in both types of class. 

Step 4: Identify the academic and 

SUPPORT UNITS THAT PARTICIPATE IN 
THESE ACTIVITIES 

The next step in building your Economic Model is to 
identify the organizational units that are involved in 
the activities and tasks. 

Documenting the organizational units involved most 
directly is only the first step. Do not miss units and 
individuals involved in supporting roles. For 
example, which units and individuals help the faculty 
member use technology in a course - training? 
Instructional development? Moving equipment to 
classrooms? Repairing projectors? Solving problems 
of connectivity for students? Purchasing software? 
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Step 5: Identify the resources these 

UNITS CONSUME IN THEIR ACTIVITIES 

Your next step is to identify the time, money, space, 
equipment, and other resources these units and 
individuals consumed to produce your output. Your 
institutional financial records system office may be 
able to provide budgeted and actual expenditure data 
for each account maintained by a unit. However, 
accounts do not always translate easily into a specific 
activity such as course development. You will need 
to cut and paste pieces of these resources into your 
model, using different weighting or allocation 
schemes. 

Obtaining data about how much time people spend 
on each activity and task (workload effort) is one of 
the more challenging elements of building a cost 
model. Not only is it quite difficult to estimate time 
spent on a task, even when keeping an hourly log 
(which almost no one is willing or able to do), faculty 
and staff may be wary of reporting their time spent, 
because they may fear the consequences of the study. 
(That is a major reason why many good studies seem 
to be self-studies: individuals and organizations who 
want this data so that they can make their own work 
more successful and less stressful on time and 
budgets.) Having reasonably accurate estimates of 
time spent on key tasks is essential to your model, so 
you will need to recruit faculty and staff cooperation 
tactfully. 


Estimating Time 

The II PI I and RIT implementations of the F'eonomie 
Model each in\ol\cd a survey of faculty. RIT 
supplemented this w ith interviews. 

At George Mason l niversity. the cost analysis 
methodology involves faculty interviews as well as 
administering a mini-version of the National Study of 
Postsecondary faculty (NSOPP) survey. In the end. though, 
the faculty workload data were suspect. In the future, it is 
recommend that faculty be asked to maintain daily activity 
logs. 

The P ITS project developed time sheets for project to help 
gather data: over the years, shifts in these sheets, and how 
they were used, reflected the changing priorities of the 
project. 


When faculty and staff own the process, they are 
much more likely to be committed to providing 
accurate data and, ultimately, to helping implement 
recommendations. We believe that studies like this 
should be designed to reduce the chances of burnout, 
to make careers more satisfying, and to make sure 
that when resources are really needed, they are 
available. It is important to assure those estimating 
how they spend their time that they are contributing 
to an ongoing effort to make their own work more 
productive and satisfying, and support procedures 
more efficiently. Emphasize that the data are not 
being used to show whether they are doing their work 
"correctly” or spending the “right” amount of time on 
different tasks. Representatives of respondent groups 
ought to be involved in the study design. They should 
be able to review and suggest changes to a report 
about their activity. 
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Once time estimates have been collected, the next 
step is to estimate the cost of an hour of 
time--a difficult task. Some models derive hourly 
rates from annual salaries and the assumption that the 
staff works 40-hour weeks. Others start with 
estimates of how many hours the staff really do work 
a week (50? 60?). Either way can lead to paradoxical 
results if one equates cost and value. Consider two 
classes of employee that work for the same annual 
salary. One group works a 40-hour week, the other a 
55-hour week. 


□ If hourly rates are defined using the 
assumption that everyone works a 40-hour 
week, then the ‘55 hour’ group can generate 
cost estimates that exceed their annual 
salaries, if one counts all 55 hours of 
activities. 

□ If, on the other hand, the model derives a 
different hourly rate for the ‘55 hour week’ 
group, then the model seems to imply that 
they are of less value because their ‘hourly 
rate’ (a fiction of the model) is less. 


Workload Matrix (sheet 2) 


Time Spent 

Faculty #1 Time 

Teaching Asst. 
#1 Time 

Staff #1 
Tune 

Staff n Time 

Total annual hours 
(2,080) for 12 months at 
40 hours per week (staff only) 


A 



Total annual hours based 
on contract (faculty only) 

A 




Activity 1 Hours 





Task A 





TaskB 





TaskC 





Total 

B 

B 



Activity 2 Hours 





Task A 





TaskB 





TaskC 





TaskD 





Total 

C 

C 



Total Activity Hours 

D=B+C 

D=B+C 



Activity hours as percent 
of annual hours 

E=D/A 





In order to keep our model simple, we classify costs under three broad categories: (1) compensation; (2) direct, 
non-personnel expenses; and (3) hidden costs. 


















Compensation includes employee salaries and all 
employer-paid benefits such as retirement, FIC A, and 
health insurance. The list of faculty and staff that 
helped produce an output is used to identify their 
salaries and benefits. The result is total 
compensation, which is then calculated against the 
percent of total workload spent on the activities being 
evaluated. 

Direct, non-personnel services include all typical 
academic and administrative office expenditures such 
as travel, equipment, copying, software, and supplies. 
These expenses are considered direct expenditures 
because they are specifically related to the output 
being evaluated. For example, the technology course 
involves the purchase of software and the traditional 
course requires copying. Or, this person may attend a 
conference to help develop the skills needed to create 
online content. Departmental expenditures for non- 
personnel services for administrative overhead are 
not considered separately, but fall under the category 
of hidden costs. 

Hidden costs , while often not identified in budgets, 
still effect how much it costs to offer an innovation 
with technology. They include, but are not limited to, 
administrative overhead, depreciation, building costs, 
and utilities. Part III of this Handbook will discuss in 
more depth how to incorporate hidden costs into your 
Economic Model. 

For now, there are three types of hidden costs to 
consider: (1) administrative overhead; (2) physical 
plant; and (3) activity-based costs related to a specific 
course, but that will not be found in a departmental 
budgeting system. 

Administrative overhead is not addressed in the 
sample model, but is discussed in the report of the 
cost methodology used by George Mason University. 
It may not be important to address administrative 
overhead at this point, unless there is good reason to 
believe that departmental/unit overhead will vary 
widely with different units and activities in the 
model. 


The example of hidden costs discussed in Part III 
includes space and equipment. Other hidden, 
activity-based costs that might be associated with the 
course could include purchasing, setting up, and 
maintaining the hardware and software for the web 
server on which the online materials are housed. 


Equipment Matrix (sheet 3) 


Computer 

Trad. 

Course 

Tech. 

Course 

Computer expenditures 


A 

Depreciation schedule/years 


B 

Depreciation per annual year 


C=A/B 

Depreciation per semester 


D=D/2 

Utilization for Course 


E% 

Utilization Cost 


F=D*E 


As you create your model, you will need to move 
through layers of these worksheets. Each time you 
examine a cost area more closely, such as equipment, 
you may find that it is difficult to fit it into an 
existing worksheet. Just create a new table for 
equipment, listing all hardware and who is using it. 
You may also want to measure the hours of usage of 
the equipment. How much of this usage is devoted to 
the activity you are evaluating? How should the 
equipment cost be amortized? This approach allows 
you to prorate equipment costs 
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Determine which accounts you need to examine 

Most financial systems in higher education involve 
thousands of account structures. An academic department 
usually has a general account to which expenditures and 
compensation are charged, as well as separate accounts for 
sponsored research. In order to identify all costs, you may 
need to examine several accounts. These are usually broken 
down into categories of expenditures, sometimes called 
object codes, such as travel, publications, and equipment. 
Individual purchases or transactions arc combined into 
object codes, based on budget, reporting, and audit 
requirements. 

At WSU and RTF. team members found it difficult to 
identify all costs because the university had no central 
repository for the kinds of data for which the team was 
looking. In addition, WSU team members found it hard t o 
explain to administrators that they were seeking 
information about monies drawn from operating funds 
rather than capital funds. The team members are hopeful 
that with more experience they will find it easier to eolleet 
cost data. 

GMU was able to use data from its Data Warehouse to 
document direet expenditures and compensation at the unit 
level. These data were shared with faculty as part of the 
initial discussion of costs, f inancial data were av ailable by 
account at the object code level. During the faeulty 
interviews, it became clear that individual faeulty members 
paid a large part of the cost of developing the technology 
themselves, without reimbursement from their departments. 


Step 6: Calculate activity costs 

You have identified your resource concerns, general 
and specific outputs, activities, and resources. You 
have identified any metrics by which you might 
further quantify costs and you understand the 
relationships between variables in terms of cost 
drivers. Now you are ready to calculate the costs of 
the various activities and tasks involved in producing 
your output. A simple spreadsheet model to account 
for several activities might look like Sheet 4, with 
breakouts for each type of cost and for each person 
involved in the activity. 
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Cost Matrix (sheet 4) 


Activity-based costs 

Faculty#l 

Teaching 
Asst. #1 

Staff#! 

Staff #2 

Total 

Salaries 





A 

Benefits (24%) 





B=A*24 

Salaries and Benefits 





C=A+B 

Activity hours as percent of 
annual hours 





D (from 
worksheet) 

Total Activity Compensation 





E=C*D 

Activity 1 hours 





F (from 
worksheet) 

Activity 1 portion of 
compensation 





G=(F/D)*E 

Activity 2 hours 





H (from 
worksheet) 

Activity 2 portion of 
compensation 





I=(H/D)*E 

Activity 3 hours 





J (from 
worksheet) 

Activity 3 portion of 
compensation 





K=(J/D)*E 







Non-Personnel Services 






Travel 






Photo copying 






Phone 






Equipment 






Software 






NPS total 





L 

Hidden Costs 






Space 






Equipment Depreciation 






Hidden costs total 





M 

Total Costs 





N=C+L+M 




Step 7: Tally the costs of all activities to arrive at your output costs 


The aggregated spreadsheet for this example would look like this: 

Summary Matrix (sheet 5) 



Trad. 

Course 

Tech. 

Course 

Equation 

Total costs (from worksheet) 



A 

Headcount enrollment 



B 

Course duration (weeks) 



c 

Meeting hours per week 



D 

Weekly student course hours 



E= 

(B*C*D) 

Cost per weekly student 
course hour 



F=A/E 


Think of your Economic Model as a pyramid 
constructed of blocks. Each block is a spreadsheet. 
The top block, or highest-level spreadsheet, contains 
data about the total cost per output for all activities 
and tasks. The base of the pyramid is made up of 
many spreadsheets, each of which analyzes costs for 


one activity and its related tasks. As you work your 
way up the pyramid, you feed or link subtotals from 
the lower spreadsheets into those at a next higher 
level. This results in a gradual aggregation or tallying 
of costs, as illustrated in Figure 1 . 
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Figure 1 

Linkage of data from lower details matrices up to top-level summary matrix. 
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In the example of costs per weekly student course 
hour for the traditional and technology sections of 
Sociology 101, possible cost drivers (factors with the 
greatest influence on total costs) include student 
enrollment, faculty workload, and other resources 
consumed. Each driver can be tweaked in the model 
to ask “what if’ questions. Build your model so that 
you can change the cost drivers and study the 
influence of these changes on cost/per student and 


cost/course. For example, you might want to stay the 
effect of increasing enrollment for the technology 
course. For the IUPUI model that examined retention, 
the evaluators looked at the effect of increasing 
retention on costs. 

In the next section, we will fill in the blanks of our 
sample model, showing more detail about each of the 
worksheets is designed and integrated. 
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PART III: THE SAMPLE MODEL 


For the purpose of this Handbook , we developed a 
fully functional, sample model. Here is the story 
behind the model: 

At a hypothetical medium-sized institution with 
20,000 students, there is a mix of doctoral and 
masters programs in arts and sciences. There is no 
Ph.D. program in sociology, but numerous 
undergraduate and master’s students. In other 
institutions this size, Ph.D. students are used to help 
to teach introductory courses. Therefore, the 
department has had to rely on full-time, ranked 
faculty to teach these classes. While it has tried to use 
more part-time faculty, it takes a lot of work by the 
chair to hire and monitor these adjuncts. In addition, 
there is a lot of turnover among the young, recent 
Ph.D., part-time faculty members because they move 
away to take better jobs. 

The dean and department chair want to reduce the 
amount of full-time faculty time involved in teaching 
introductory sociology courses. This will keep the 
faculty happy, letting them concentrate more on 
research and working with masters students; but how 
will the department be able to keep up with student 
demand? The idea of using technology and serving 
more students has been discussed. The administration 
would like to know what it costs to teach introductory 
sociology the old way versus using Internet 
technology? Could this type of course accommodate 
more students and an increased faculty to student 
ratio? 

Since there is vocal discussion among faculty about 
whether online courses are as academically rigorous 
as their traditional counterparts, the chair does not 
want to offer the course entirely online. Nor does she 
have much money to spend on developing course 
content for the web. A new assistant professor is 
interested in teaching the technology course. A 
tenured, associate professor teaches the traditional 
counterpart. 


The resource question of interest is: 

What is the difference in total cost per section 
between a traditional section of Sociology 101 that 
meets twice a week for fifteen weeks and a section 
that meets only a few times and does most of its work 
on the Internet? What is the total cost per weekly 
student course hour? A lower cost in each measure 
will be interpreted as better. 

The model that is presented was designed to compare 
this specific set of courses and the results are only 
useful for this one institution at a specific point in 
time. 

The activity matrix documents the differences and 
overlap in activities between the traditional and non- 
traditional sections. While the traditional instructor 
must be physically present in the classroom and 
maintain office hours, the online instructor trades 
these duties for those of setting up the website, using 
Cold Fusion software to develop online testing, 
monitoring the listserv, and responding to student 
email. 

One of the most critical steps in cost analysis 
involves the process of defining activities and 
breaking these up into tasks. This also helps to 
identify specific purchases or additional human 
resources used. In the technology course, there is an 
additional task that requires purchasing Cold Fusion 
software. 

Note from the activity matrix that a lot more time is 
spent in the technology class on preparing materials. 
In the future, while there will be some new electronic 
material and updating necessary, the total amount of 
time spent by faculty may actually decline. This is a 
cost driver to be tweaked with future iterations of the 
model. 
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Activity-Task Matrix (sheet 1) 


List of Tasks/Activities 

Traditional 

Class 

Faculty 

Hours 

Technology 

Class 

Faculty 

Hours 

Equation 

Activity 1 - Prepare Course 






A. Coordinate with dept, chair 

X 

1 

X 

1 


B. Create syllabus 

X 

4 

X 

4 


C. Prepare electronic version 



X 

20 


D. Update course materials 

X 

8 

X 

8 


E. Develop website 



X 

40 


F. Find/maintain web server 



X 

6 


G. Develop online materials 



X 

80 


H. Develop quizzes 

X 

8 

X 

8 


I. Learn Cold Fusion software 



X 

15 


J. Build interactive quizzes with 
Cold Fusion software 



X 

15 


K. Develop graphics 

X 

8 

X 

8 


L. Put graphics online 



X 

2 


M. Set up listserv 



X 

2 


N. Set up email filters for class 



X 

2 


Total for Activity 7 


29 


211 

A 







Activity 2 - Teach Course 






A. Teach class 

15 weeks x 2 
meetings & 1.5 
hrs each 

45 

1 meeting for 
start, mid-term, 
and final exam 

6 


B. Office hours 

6 hrs per week x 
15 weeks 

90 


0 


C. Monitor enrollment 

X 

2 

X 

2 


D. Monitor listserv 



X 

30 


E. Respond to student email 



X 

30 


F. Respond to student phone calls 

X 

2 

X 

8 


Total for Activity 2 


139 


75 

B 







Activity 3 - Grading 






A. Administer quizzes 

X 

2 




B. Grade quizzes 

X 

2 




C. Report quiz grades 

X 

2 




D. Report quiz grades online 



X 

1 


E. Administer exams 

X 

2 

X 

2 


F. Grade exams 

X 

8 

X 

8 


G. Report exam grades 

X 

1 

X 

1 


Total for Activity 3 


17 


12 

c 

TOTAL for all Activities (A+B+C) 


185 


298 

D=A+B+C 
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Better understanding of faculty workload issues is 
another byproduct of this cost analysis process. The 
following chart documents overall faculty effort. 
Both classes have a teaching assistant. While the 
traditional class uses the teaching assistant to grade 
quizzes and monitor enrollment, the technology class 
uses her to put quiz material online. For the 
traditional class, a departmental program support 
technician is used to distribute grade and enrollment 
reports and to answer general student questions 
before the course starts. The technology class 
requires staff help from two people in academic 
computing support to set up space, install Cold 
Fusion, and monitor security on a PC with NT Server 
running the web server software. 


Faculty Workload Matrix (sheet 2) 


Traditional Class 

Total Hours 

Faculty #1 

185 

Teaching Assistant #1 

150 

Staff #1 (office clerical) 

10 



Technology Class 


Faculty #1 

298 

Teaching Assistant #1 

10 

Staff #1 (computer support center) 

20 

Staff #2 (computer support center) 

20 


Comparing the two sections, the faculty member 
using technology spent more time on developing the 
online class and less time on direct teaching. Overall, 
the faculty data show that this person spent 248 hours 
on specific course-related tasks, compared to the 185 
worked by the other faculty member. An activity log 
may need to be maintained daily by faculty and staff 
during several weeks of the semester to capture all 
hours of faculty and staff time spent on a project. 
This would allow the model user to better document 
the number of hours spent on each task. 

The first resource matrix builds upon the faculty 
workload data. 
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Resource Matrix #1 (sheet 3) 


Faculty Costs 

Trad. Faculty 

Tech. Faculty 

Equation 

Total 9 month salary 

55,000 

40,000 

A 

Benefits (university contribution) 

24% 

24% 

B 

Benefits amount 

13,200 

9,600 

C=A*B 

Total compensation 

68,200 

49,600 

D 

Hours worked per week 

50 

60 

E 

Total 9 month hours worked 
(9 months of 4 weeks) 

1800 

2,160 

F=E*36 

Hours spent on class activities 

185 

298 

G 

Percent for section 

10.28% 

13.80% 

H=G/F 

Faculty Costs per section 

$7,009 

$6,843 

I=H*D 





Other Personnel Costs 

Trad. Course 

Tech. Course 

Equation 

Teaching Assistants (each paid 
$4,000 per semester, assisting with 
4 courses, of which this is 1). 

$1,000 

$1,000 

A=(4, 000/4) 

Staff 1 Time 

10 hrs 

20 hrs 

B 

% Annual Time (2080 hrs) 

0.48% 

0.96% 

C=B/2080 

Salary 


35,000 

D 

Benefits (24%) 


8,400 

E=D*.24 

Salary/Benefits Total 

31,000 


F=D+E 

Cost per section 

$149 

$417 

G=F*C 

Staff 2 Time 

0 

20 hrs 

H 

% Annual Time (2000 hrs) 

0.00% 

0.96% 

I=H/2000 

Salary 

0 

35,000 

J 

Benefits @ 24% 

0 

8,400 

K=J*.24 

Salary/Benefits Total 

0 

43,400 

L=K+J 

Cost per section 

$0 

$417 

M=L*I 

Total Other Personnel Costs 

$1,149 

$1,835 

N=(A+G+M) 

Total Personnel Costs 

$8,158 

$8,678 



When all personnel costs are accounted for, the technology course takes more faculty time and relies on computing 
staff help to develop the web server and listserv. The tenured faculty member costs more than the new assistant 
professor (who is eager to use technology and hopefully become established). 

Departmental costs are broken out for those non-personnel services that can be directly attributed to each section. 

The traditional section involved some photocopying. For the technology course, the instructor purchased a software 
package for building websites and a digital drawing pad for creating graphics. Since the drawing pad is low cost, it 
will not be depreciated like other equipment. 
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Resource Matrix #2 (sheet 4) 


Sociology Department 
account #1-67382 

Trad. Course 

Tech. Course 

Equation 

Total Compensation 

$8,158 

$8,678 

A 

Direct Non-Personnel Services 




Photo copying 

100 


B 

Drawing Pad 


150 

C 

Software 


500 

D 

NPS subtotal 

100 

650 

E=B+C+D 

Total Direct Costs 

$8,258 

$9,328 

F=A+E 


Hidden costs. One of the largest areas of hidden costs 
is space. Both classes are dependent on some number 
of meetings in a physical classroom. How should this 
use of the building be accounted for? Should building 
maintenance and depreciation be built into the 
equation? 

The traditional analysis of classroom utilization 
sometimes fails to account for creative use of space 
and for the true cost of this precious capital outlay 
resource. The National Commission report adopted a 
model by Gordon Winston that incorporates the 
concept of ’’opportunity costs’’ (Harvey et al, 1998). 
Opportunity costs estimate the value of missed 
’’opportunities" in which the institution could have 
rented the space at fair market value. 

If you decide to estimate space costs, there are a 
number of sources to help you calculate the 


cost-per-square-foot figure. These include projected 
annual costs for new construction, the cost of renting 
comparable space in a commercial facility, or figures 
derived from capital outlay financial records. You 
will need to decide what source is most appropriate 
for your situation. The Gordon Winston model also 
includes a simple method for estimating building 
depreciation costs, estimated at 2.5%. The annual 
2.5% depreciation rate is applied to the replacement 
or market value of buildings. 

To allocate the building cost for the example model, 
we need a supplemental worksheet. Fair market 
rental prices are used for the model. Data on actual 
replacement costs for the building and depreciation 
are not needed, since these are already included in the 
commercial rental price. 


Space Matrix (sheet 5) 


Space 

Trad. 

Course 

Tech. 

Course 

Equation 

Cost per sq. foot 

$2 

$2 

A 

Number of months rented 

4 

4 

B 

Semester cost per sq. foot 

$8 

$8 

C=A*B 

# Sq. feet 

1,000 

1,000 

D 

Semester cost for rental 

$8,000 

$8,000 

E 

# Weeks of class meetings 

30 

3 

F 

# Hours per meeting 

1.5 

3 

G 

# Class hours in room 

45 

9 

H=F*G 

All class hours meeting in 
room (from schedule) 

225 

225 

I 

% Utilization 

20.0% 

4.0% 

J=H/I 

Cost for the section 

$1,600 

$320 

K=J*E 
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First, we take the cost-per-square-foot figure and 
multiply it by the room’s number of square feet to 
arrive at the total opportunity cost of the room, 
prorated by the number of months of the rental (four 
since it is needed for a semester of 15 weeks). Next, 
we look at the use of the room. Both courses meet in 
the same room. The traditional course meets two 
times a week, ninety minutes a day, for fifteen weeks. 
The 30 class sessions take up 20% of the room's 
utilization, calculated by listing all classes that meet 
in the room and totaling the hours they meet. The 
technology course only meets three days out of the 
entire fifteen weeks, or four percent of the room's 
total space utilization. 

The registrar's staff that schedules rooms produces, as 
part of the scheduling process, a series of reports with 
course data by room, broken out by time and day of 
week. You may want to request a copy of these 
reports to help build your space matrix. These data 
should be verified by talking with the faculty and 


staff members involved in the activities under study. 
Many last minute room changes never show up in the 
room scheduling system. 

Another area of hidden costs is depreciation of 
equipment. In this case, the only equipment consists 
of the Windows 2000 Server computer which houses 
the web server and Cold Fusion application server 
and the computer used by the technology course 
faculty member to develop online materials. The 
traditional course's faculty member has a computer, 
but does not use it for the course, only to check 
email. 

To calculate equipment depreciation, the purchase 
price of the computers is amortized over their 
lifetime and adjusted based on an appropriate 
amortization schedule. This spreadsheet might look 
as follows: 


Equipment Matrix (sheet 6) 


Computer 

Trad. 

Course 

Tech. 

Course 

Equation 

Pentium 200 w/ 64 MB Ram 
and 4 GB hard disk. 
Purchased 1997 

0 

2,000 

A 

Utilization for Course 

0 

75% 

B 

Utilization Cost 

0 

1,500 

C=A*B 

Dual Pentium II 350 w/ 256 
MB Ram, SCSI, RAID 5, NT 
Server 

0 

8,000 

D 

Utilization for Course 

0 

10% 

E 

Utilization Cost 

0 

800 

F=D*E 

Total Utilization Cost 

0 

2,300 

G=C+F 

Lifespan 

0 

5 years 

H 

Depreciation per acad. Year 

0 

460 

I=G/H 

Depreciation per semester 

0 

230 

J=I/2 
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Now that the various building blocks of the cost model have been assembled, the key cost components need to be 
aggregated into a single worksheet. 


Summary Matrix (sheet 7) 


Subtotals 

Trad. Course 

Tech. Course 

Equation 

Compensation & Direct NPS Costs 

$8,258 

$9,328 

A 

Space Opportunity Cost 

1,600 

320 

B 

Computer depreciation 

0 

230 

c 

Total Costs per Course 

$9,858 

$9,878 

D=A+B+C 

Total Student Headcount 

35 

100 

E 

Total weekly student course hours 
(3 hours per student per week x 1 5 
weeks = 45 weekly hours per 
student) 

1,575 

4,500 

F=E*(45) 

Cost per weekly course hour 

$6.26 

$2.20 

G=D/F 


Cost Drivers. What are the cost drivers associated 
with this sample model? Let us look closer at the 
variables that may change over time. These include 
the number of hours spent by faculty on each activity, 
faculty compensation, student enrollment, computer 
use, space use, and the performance measure 
calculation of weekly student course hours. 

What are some of the ways in which the model can 
be tweaked by the sociology department chair and 
others to examine cost constraints and issues? 

❖ In the future, the number of hours spent on the 
activity of online content development will 
decrease. If this time is not put into another 
activity, the overall number of hours spent by 
faculty will decrease. This will lower the portion 
of faculty compensation used and therefore 
decrease costs. 

❖ Instead of using a tenured associate professor 
who is paid $55,000 a year, the chair can assign 
the traditional course to another new assistant 
professor to teach at $40,000. This will decrease 
the faculty costs of the traditional course by 
$15,000 (27.3%). 

❖ The enrollment of the technology course could 
be increased with the assumptions that the 
faculty could use the listserv and email to handle 
more students and that the online materials are 
already created. This would increase the number 
of weekly student course hours and decrease the 
performance measures results of cost per weekly 
student course hour. 


❖ It is possible to use an existing computer, 
therefore decreasing depreciation costs. If the 
instructor teaching the traditional course gets a 
computer and uses it for course email, these costs 
go up. 

❖ If the technology course meets more often during 
the semester, the space costs go up. 

❖ In the future, computer center support staff may 
not be needed for the technology course. This 
would decrease the personnel costs by $868. 

❖ The Cold Fusion software and drawing pad will 
not be purchased again. Next time the course is 
offered, there will be less spent on direct, 
non-personnel services. 

Findings 

Based on the structure of the example model and the 
details of the matrices, we can highlight some of the 
model's findings. 

In terms of the first performance measure, the cost 
per course for the traditional course ($9,858) is 
almost equal to that of the technology course 
($9,878). The faculty member's time in the 
technology course is merely, displaced, from time 
spent in the classroom to time spent monitoring the 
listserv, updating online content, and responding to 
emails. However, when cost per weekly student 
course hour is used as the performance measure, then 
the larger headcount enrollment that may be 
accommodated with the online course significantly 
reduces the section cost. The online cost per weekly 
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course hour is almost a third of the traditional course, 
$2.20 instead of $6.26 per weekly student course 
hour. 

Assumptions about the appropriateness of increasing 
enrollments with technology courses need to be 
raised. The literature on learning styles and 
philosophy of education has much to say about the 
efficacy of serving more students with technology. 
This is one reason the Flashlight Inventory is so 
useful, for it gives institutions a complete set of tools 
for examining critical questions about the cost- 
effective and pedagogically sound use of technology. 

Variation in methodology. You will note that there 
are many possible variations within the models 
described in this Handbook. When you report the 
findings of your Economic Model, it is essential to be 
explicit in describing what factors or “tweaks” 
affected your costs. Faculty workload is one 
component of the model that will vary widely, 
depending on the circumstances and on the way in 
which these data are collected. 

It is important to pick apart your assumptions about 
the model, because others will certainly do this, so be 
prepared for their ideas. 

Variation in costs. Once you build your model, 
choose the outputs and measures, and gather the data, 
you may or may not be surprised by the results. If 
you are surprised, do not take the data at face value. 
Perhaps there is an error in the model, or odd data 
shows up for one year, or you are using an estimated 
cost driver. Be sure to check all of the cost drivers in 
the model to verify that they are having the intended 
effect on the equations you designed. Run the model 
with another year's worth of data and see if you come 
up with the same results. If they are different, ask 
yourself why. 

Other benefits of modeling. There is a byproduct to 
breaking down these complex activities into tasks - 
documentation of the steps for future innovations 
with technology. The task lists become a flowchart 
for replicating the use of technology, using the same 
types of steps for each subsequent innovation and its 
use. If someone is interested in a specific type of 
technology, the list of activities and tasks, along with 
previous faculty workload data is an invaluable 
blueprint that will help save time and money and 
prevent reinvention of the wheel. 

The information gathered about specific faculty, 
staff, and student workload, through surveys, 
interviews, or methods designed to document time on 
task, is a valuable tool as well. This helps 
administrators and faculty better understand faculty 
roles and the tensions of competing priorities for their 


time. As the dominant expenditure in higher 
education, personnel costs need to be questioned as 
part of any modeling of resource use. The faculty 
workload data gathered with this process are 
important, in and of themselves, for decision-making 
about resources. 

Similarly, the work done in modeling on defining 
performance measures for outputs is another helpful 
by-product of cost analysis. Many institutions are 
engaged in a broad dialogue about performance 
indicators relevant to their mission. These measures 
often boil down to ways to measure costs, efficiency, 
effectiveness, and quality - all variables that can be 
incorporated into your modeling. 

Developing the modeling process 

In writing this book, we had to decide between: 

a) developing a simple (and perhaps simplistic) 
model that would be relatively easy to 
understand and more flexible for a variety of 
studies, or 

b) a more elaborate model that would include the 
subtleties but that might be harder for novice 
model builders to understand and that would 
have focused on just one type of cost study. 

We chose the former course. 

One important component of cost models that is not 
explored in this Handbook is revenue, and 
technology use can sometimes increase revenue (e.g., 
additional enrollment-related revenue, grants for 
innovation). Revenues are considered, however, in 
the George Mason case study in Appendix B. How 
much money was generated by the technology 
innovation you are evaluating? It appears on the 
surface that the technology course, which taught 100 
students, generated more revenue. This basic 
enrollment equation is not always true. Financial aid, 
especially the institutional practice of net tuition 
discounting, needs to be examined in order to make 
these kinds of revenue estimates. As the IUPUI case 
study points out, there is also a revenue factor 
implied when you use performance measures related 
to retention and attrition. Your school saves money 
by keeping students enrolled. There is a tremendous 
cost to students as well for dropping out, effecting 
long-term income potential (another hidden revenue 
generator). 

There may be more hidden costs that the model does 
not describe which you will discover over time For 
example, in working with some online course 
providers to develop content, there is a fee per 
student charged by the vendor to the school. The 
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school does not have to staff and support content 
development because this has been outsourced or 
privatized. However, the revenue loss may be 
temporary, as the vendor may be able to turn around 
and market the course to a larger audience, offsetting 
the initial high development cost with royalties to the 
institution. 

One exciting possibility is to use “cost” models to 
explore ways to reduce stress in jobs and make 
activities more fulfilling for the people who do the 
work. So far, many people see cost models as a 
threat, and (often) rightly so. Traditional cost models 
treat working time as a cost, something to be reduced 
in order to keep students and alumni out of debt 
while reducing demands on taxpayers and donors. 
That is an important thing to do, in an era when more 
people need to learn while fewer people (per learner) 
are available to pay taxes and give gifts to colleges. 
At the same time, however, most staff members have 
full-time jobs. From their point of view, the cost 
model appears to have only one potential impact if its 
goal is ‘time saving’: do their jobs continue to exist 
or are they to be eliminated? We suggest that it is 
more constructive, both for the purposes of cost 
savings and for the staff involved, if models are 
designed in order to help work processes become 
more effective, efficient, and fulfilling. The 
University of Pennsylvania case study on engineering 
laboratories in Appendix II is a step in this direction. 
Pope and Anderson describe a modeling process that 
helped reduce time spent by faculty and staff on 
burdensome activities such as grading, record 
keeping, and training in how to operate lab 
equipment while maintaining time spent helping 
students learn. Future models ought to focus on 
reorganizing activity so staff members can see that 
their work is successful, is efficient in its use of the 
time, and is less of a strain on budgets. If you do such 
a study, or know of one, we hope you will suggest 
that it be included in the next edition of this 
Handbook ! 


Conclusion 

This Handbook has presented an Economic Model 
methodology for conducting cost assessment studies 
of technology in teaching and learning. Part I 
explored the general approach to cost analysis, 
including the classic NACUBO indirect cost recovery 
approach and recent work funded by the Andrew W. 
Mellon Foundation, the Annenberg/CPB Project, the 
Alfred P. Sloan Foundation, and the Pew Charitable 
Trust. The organizational constraints and issues that 
face institutions when they initiate this type of cost 
study were highlighted, such as the necessity for a 
team process. Part II outlined the basic building 
blocks for creating this kind of model. Inputs, units, 
and resources are used to calculate outputs and 
performance measures to answer a set of resource 
questions. In part III, a sample model was analyzed in 
detail with specific worksheets that document 
activities and tasks, workload, compensation, direct 
non-personnel costs, and hidden costs such as 
equipment and space. 

The methodologies listed here are relatively easy to 
apply, although the data are sometimes difficult to 
gather. While modeling may appear to provide 
answers for the questions you are interested in, there 
are always things to pick apart or tweak. The 
dialogue that follows at your institution about the 
model and its cost evaluation is just as important as 
the results, if not more so. Be prepared for and 
welcome criticism. It is all part of the change process 
for innovation. It is critical that institutions begin to 
discuss the issue of costs related to technology and to 
address resource concerns in this way. This 
Handbook is a valuable first step in this process. 

This Flashlight Cost Analysis Handbook will 
continue to evolve. The authors welcome hearing 
from you about your own case studies, including how 
you developed your methodology, any results you 
wish to share, and how the results have influenced 
decision-making. Look for the next edition of this 
Handbook , as the authors continue learning with you 
about modeling resource use in teaching and learning 
with technology. 
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GLOSSARY 

Activity - Specific jobs, roles, duties, or functions 
performed by individuals and/or units. Examples 
include course preparation, teaching, and enrollment 
management. 

Activity-Based Costing (ABC) - A method of 
evaluating costs by the basic types of activities 
involved. These activities may cut across 
organizational boundaries and include many types of 
expenditures, including some hidden expenses that do 
not normally appear on operating budgets. 

Cost drivers - Variables in a model that have a 
substantial impact on total costs. Examples include 
average enrollment per course, the number of hours 
typically spent on an activity, and the amount of 
space needed for each additional unit of activity. 
Altering these cost drivers in a model is one way of 
exploring strategies for reducing the cost of achieving 
a particular level of performance (e.g., changing the 
assumption about number of dropouts from a course 
and the influence of this cost driver on cost per 
student completing the course). 

Macro-costing - The process of examining 
aggregate costs at the easiest-identified unit level. 
The standard accounting system structure is used, as 
in indirect cost recovery models. A variety of 
complex allocation schemes is possible to allocate 
different tiers of costs. This methodology lets the user 
develop cost-sharing calculations for hidden costs 
such as administrative overhead at the department, 
dean, and university-wide level; student services; 
utilities; and physical plant. 

Metric - A way of measuring an activity, task, or 
specific cost. 

Micro-costing - A way to evaluate costing from the 
ground up by examining any and all costs associated 
with a project or program (cost center or focus). The 
idea is to document the full cost of the activity or 
process. Activity-based costing may be used as part 
of this process. 

Performance measure - The specific output 
analyzed by a model. It measures a general output, 
such as producing a CD-ROM. An example might be 
the number of students who use the CD. Other 
measures used in this Handbook include weekly 
student course hours 


Responsibility Centered Management - A way to 
promote responsible management practices by 
evaluating and assessing costs and programs. 
Activity-based costing is often used as part of this 
process. 

Task - Each activity can be broken down into 
component activities (“tasks”)- Examples of tasks 
that are part of the activity of course preparation 
might include reviewing the syllabus, selecting 
readings, and revising the sequence of materials to 
meet course objectives. 

Weekly student course hour - An example of a 
performance measure used in the Handbook's 
example model. This calculates the number of hours 
in which students are enrolled to “sit” in a course or 
activity. For a course, this is often three hours per 
week for fifteen weeks. This differs from student 
credit hour (SCH), which is the number of credits 
awarded for passing a course. 
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CASE STUDY 1 


Indiana University Purdue University Indianapolis 
Cost Implications of Technology Use in the 
IUPUI English Department’s Core Writing Curriculum 

IUPUI Economic Model Office 
Indiana University Purdue University Indianapolis 
24 June 1996 


Revised June 1999 with approval for the purposes of this handbook 


Abstract: This study of costs is a component of the Annenberg/CPB Flashlight Program in 
which IUPUI has joined four other institutions (Washington State University, Rochester 
Institute of Technology, The University of Maine, and Maricopa Community Colleges) for the 
development of a set of evaluative instruments which educational institutions can use to monitor 
and assess their usage of technology in instruction. 


Background 

In March 1995, the Economic Model Office at IUPUI 
undertook a comparative study of the costs associated 
with providing introductory instruction in English 
composition, given two different instructional 
formats: 

❖ utilizing a traditional classroom; 

❖ utilizing networked computer classroom (NCC) 
instruction in a modified classroom containing a 
series of networked computer workstations. 

Two key issues are the focus of this cost analysis of 
the English composition courses: 

1. Does it appear to be more cost effective (in 
operating costs) to offer a program where 
students participate over networks than one 
where students come to traditional university 
facilities for classes? 

2. If the university’s use of technology results in 
increased retention, what are the economic 
implications of that improvement (if retention 
data can be obtained from some source)? 

Summary of Findings 

With the limitations of available data and 
confounding variables taken into consideration, it 
was found that for the two courses analyzed it did not 
appear to be more cost effective to employ 
technology in the given way to instruction. Further, 
the application of network technology was associated 


with a negative effect on retention as defined, and 
accordingly magnified the negative economic impact. 

Description of Study 

IUPUI English curriculum core writing courses. 

The English department core writing course 
curriculum was chosen for the comparative cost 
analysis for the Flashlight Program because the 
majority of its courses are offered in both traditional 
and networked computer classroom (NCC) formats, 
and a sequence of writing courses in these formats 
was available for study. The composition by 
computer / networked computer classroom course 
option has been available since 1986 on a limited 
basis with one computer classroom, and on an 
expanded basis with the upgrade and addition of a 
second computer classroom in 1993. The IUPUI 
Writing Coordinating Committee assumes an 
equivalent educational outcome from both 
instructional formats. 

Networked computer classroom courses utilize two 
classrooms located in Cavanaugh Hall, rooms 323 
and 425. Each computer classroom is 672 square feet 
in a 28’ x 24’ room. Room 323 has no windows and a 
blackboard. Room 425 has windows and a dry-erase 
board. Each computer classroom has 22 student 
desktop computers connected to two laser printers, an 
instructor desktop computer connected to an Elmo 
copy stand camera system, and an overhead video 
projector. The computer classrooms are not 
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connected to a local area network beyond the 
immediate classroom. 

The NCC course sections are split into two parts over 
two class days. One day the class is held in the 
computer classroom at the desktops and the other day 
the class is held in a traditional classroom. (Classes 
meeting once a week have both a traditional 
classroom and a computer classroom assigned, and 
move between the two rooms at the instructor’s 
discretion.) This arrangement allows for the doubling 
of opportunities for students to use computers. Twice 
as many sections have access to the computer 
classrooms, but each student has access to computers 
for only half the class meetings. The computer 
classrooms are available for individual student use 
whenever a computer is available, even if another 
class is being conducted. The computer classrooms 
always have consultants available for individual 
assistance when they are open; there is usually an 
hour or two of open classroom time each week, and 
any student can drop in and use the computers then. 

The traditional instructional format classrooms are 
typically standard classrooms with blackboards 
located in both Cavanaugh Hall and the Mary Cable 
Building. The off-campus courses are conducted 
throughout the city of Indianapolis - from high 
school classrooms to shopping mall training rooms. 
All off-campus English course offerings are taught in 
the traditional instructional format. Traditional 
sections typically have a capacity of 27 students. 

Data for this study. 

Three key data sources from the 1994-95 
school/fiscal year were used in this study. Core 
accounting data regarding resource costs were 
derived from institutional databases accessed by the 
university’s Financial Information System (FIS). 
These resource costs included payroll, benefits, 
supplies, equipment, and other pertinent costs. 

The second data source, from which enrollment and 
retention data was collected, was institutional 
databases maintained by the Office of Information 
Management and Institutional Research. While these 
data were not sufficiently extensive to address a wide 
range of questions regarding retention, approximate 
data regarding successful completion of courses were 
available. 

The third data source was the English department 
faculty. They provided information regarding effort 
devoted to each of their instructional activities, and 
assistance in interpreting the findings. 


W131 - 231 course selection. 

This report will focus on an analysis of the costs 
associated with two specific courses: W131, 

Elementary Composition I, and W231, Professional 
Writing Skills. Given the available data for 
comparative purposes, these two courses were best 
suited to this study of costs across the traditional and 
NCC formats. 

Definitions 

Given the complex nature of cost issues, a few terms 
will be defined before presentation of methodology 
and findings. 

Course W13 1 : Elementary Composition I (first 
year composition) 

This course is designed for students to illustrate in 
portfolios the ability to perform a variety of writing 
tasks from personal writing to academic writing. The 
course starts with a list of course goals that drive 
student’s choices about what to include in their 
portfolios. In their portfolios, students present an 
argument that justifies the choices made and 
illustrates their growth as writers. 

Course W231: Professional Writing Skills 

This course fulfills a second writing class 
requirement for students in the Schools of 
Environmental and Public Affairs, Allied Health, 
Social Work, and Business, among others. With a 
grade of C or better in W131 or a comparable 
composition class as a prerequisite, students are 
introduced to writing in nonacademic settings, 
particularly report writing. W231 focuses on the 
development of research skills that will be of value 
not only in the work place but also in upper-level 
courses in the student’s major area of study. 

NCC format of instruction 

The networked computer classroom (NCC) 
instruction format represents the application of 
technology to instruction using a series of 
interconnected computers within a particular 
classroom. This is not an implementation of “distance 
education”, since the students work together at the 
same time, in the same classroom, using software 
much like common text editors. Unlike the traditional 
classroom, however, students compose their work 
during class and can interactively comment on each 
others’ work in progress. 
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Note: Within the model, NCC courses are identified 
as “CAI” (computer assisted instruction). 

Operational cost 

Operational cost is the total cost of conducting the 
course. This figure includes all course sections within 
the given format (NCC or traditional), without regard 
to the number of students who enroll or complete the 
course. These costs would be expected to vary among 
courses and formats as a result of the different 
activities conducted, differing levels of effort 
required, and different resources consumed. 

Retention 

Retention can be defined in a variety of ways. From 
an institutional perspective, retention can be viewed 
as completion of a degree. From a student’s 
perspective, it can be viewed as successful 
completion of academic goals, whether or not they 
lead to a degree. Persistence, often used 
interchangeably with retention, is sometimes used to 
refer to a student completing a course and returning 
the subsequent semester for more coursework. The 
definition used in this document was constrained by 
the availability and use of data for the pilot year of 
this project. Thus for purposes of this report, 
retention will be defined as successful completion of 
a course. 

Cost per student 

Cost per student will be specified in two ways: 

❖ based on students enrolled in a specific course, 
per instructional format (traditional v. NCC) 

❖ based on students successfully completing a 
specific course, per instructional format. 

The difference in cost per student enrolled and cost 
per student successfully completing a course will be 
used as a measure of retention for purposes of this 
report. The “successful completion” count is defined 
as the total initial enrollment for the course (by 
format) minus the drop / withdrawal / fail (DWF) 
count. The DWF count is derived from institutional 
data that identifies those students, by course and 
instructional format, who either: 

❖ complete the course with a grade of C- or less, or 

❖ withdraw prior to completing the course. (This 
count is taken one week after classes begin to 
allow for initial volatility in enrollment.) 


Methodology 

The Economic Model Office at IUPUI utilizes a 
methodology called activity-based costing to 
determine total costs associated with the outputs 
produced by academic departments and 
administrative support offices. Activity-based 
costing, unlike traditional fund accounting, traces 
resource use throughout the analysis of activities 
required to produce outputs, then to the outputs 
themselves. The resulting model is not an attempt to 
determine faculty effectiveness; it is simply a current 
“picture” of the economic aspects of the subject 
outputs. 

Factors considered 

The academic outputs analyzed in this study included 
the following core writing courses: 

❖ W001 Fundamentals of English (developmental 
writing) 

❖ W131 Elementary Composition I (first year 
composition) 

❖ W132 Elementary Composition II (research and 
argumentation) 

❖ WHO Elementary Composition I / honors (first 
year composition) 

❖ WHO Elementary Composition II / honors 
(research and argumentation) 

❖ W23 1 Professional Writing Skills 

Note: E010 Access to Writing (pre-developmental 
writing) - a course within the core writing 
curriculum - was not considered since it is not 
offered in the computer classroom format. 
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The activities that the faculty members associated 
with these outcomes include: 

❖ class preparation 

❖ class instruction 

❖ office hours 

❖ assessment 

❖ advising 

❖ orientation / workshops 

❖ other. 

Resource costs examined for these activities 
included: 

❖ faculty effort 

❖ computer classroom consultant effort 

❖ facilities 

❖ computer hardware 

❖ equipment and supplies. 

Administrative overhead is not considered in this 
study, given that it is assumed to be equivalent for 
both instructional formats. Full-time faculty effort is 
self-reported data and the cost of their effort is based 
on this data. It should be noted that people can make 
mistakes in self-reported data about how they spend 
their time; therefore accuracy of these data is 
reviewed by other faculty representatives prior to use 
of the model. A weighted average method was used 
to reflect the impact of associate faculty on average 
course costs. This method derives an average amount 
of effort as reported in the survey, and applies that 
figure as weighted by the number of faculty within 
the specific labor group. (Associate faculty members 
are part-time, non-tenured instructors who are 
employed on an as needed basis and paid on a 
per-course-taught basis.) 

Process 

In May 1995, about half of the surveyed faculty (18 
of 40) completed questionnaires that allowed them to 
estimate the proportion of their effort during a 
semester spent in activities associated with the 
presentation of the six core writing courses. The 
surveyed faculty members were selected as a 
representative group of the faculty who teach core 
writing courses and who were most likely to respond, 
since the summer period had just begun. There were 
77 associate faculty and 19 full-time faculty who 
taught core writing courses during the period studied. 
The returned surveys contained responses for W001, 


W131, W132, and W231, the four courses that most 
IUPUI students take. Questionnaire responses were 
combined with financial data on compensation, 
facilities, supplies and computing costs provided by 
the school business officer and staff to construct an 
activity-based costing model. The model presents in 
detail the faculty effort and other costs associated 
with each of the core writing courses. 

The number of NCC sections has increased each 
subsequent school year since the inception of NCC, 
and so affects the data available for analysis. 
Unfortunately, a cost analysis through the entire 
curriculum is not possible due to the fact that W132 
was not offered in the NCC format during the 
1994-95 period. The variation of course formats 
through the curriculum, coupled with the non- 
traditional student population, also precluded a 
complete curriculum analysis. Therefore, the analysis 
detailed in this report is focused on the courses W 1 3 1 
and W231 because these courses had the greatest 
amount of useable data and opportunity for 
application of the cost model methodology. 

Findings 

Operating costs 

The cost model analysis revealed the following 
operating costs on a per-student basis for the period 
1994-95. Component cost contributions are detailed 
in the model. 

The model reveals that for these two courses shown 
(utilizing traditional and NCC/CAI formats) it does 
NOT appear to be more cost effective on a cost per 
student enrolled basis to offer a program where 
students participate over networked computers in a 
classroom than one where students come to 
traditional university facilities for classes. 

Completion rate 

Retention data regarding successful completion of the 
two courses reveals a higher successful completion 
rate for traditional as opposed to NCC/CAI students 
for both courses. The difference, however, is not 
statistically significant based on the chi square test. 

Retention cost impact 

After the completion rates are factored in to 
determine the cost per student comparison between 
enrollment and successful completion costs, the NCC 
format remains more expensive. The CAI version is 
W131 is 78% more expensive than the traditional in 
costs per enrolled students and 86% more expensive 
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in costs per completed student. W231 is even more 
expensive, 100% more expensive than the traditional 
in costs per enrolled students and 128% more 
expensive in costs per completed student. 

In summary, the university’s use of technology in 
these two courses is associated with decreased 
retention , thus the economic implications are 
negative for the two courses studied. Note this study 
does not consider whether the high differential costs 
were inevitable, or simply one of the results of not 
exploiting the strengths of technology in instruction 
by reengineering the instructional process . 

“What if’ analysis 

The value of the cost model architecture is its ability 
to perform “what if’ analysis to assist in decision 
making. Using the same structures defined above, 
one might ask what a scenario closer to the “break- 
even” point (cost neutral) might look like. 
Accordingly, the following data might be assumed as 
objectives for program enhancement resulting in 
more favorable cost outcomes. 

First, we might assume total cost and enrollment hold 
stable, such that the cost per student enrolled may 
still be higher for the NCC/CAI course. 

Secondly, however, we might assume that improving 
our teaching practices using the NCC/CAI 
classrooms will result in higher successful 
completion rates, such that all students enrolled in the 
NCC/CAI courses successfully complete them. 

What then would be the resulting cost impact? Once 
the completion rates are factored in to determine the 
cost per student comparison between enrollment and 
successful completion costs, it becomes clear that 
increased successful completion rates can have a 
dramatic cost impact. While not quite matching the 
cost per student successfully completing the course 
using the traditional instructional format, the net 
increase in cost is significantly narrowed over the 
previous scenario. 

One might yet assume some ability to reduce the total 
cost per course for the NCC/CAI courses, such that 
the resulting cost per successfully completing student 
would be lower than that for traditional instruction. In 
short, the use of technology and its impact on 
retention can have favorable effects on net costs 
per successfully completed student, even when the 
total course costs are higher for the courses that use 
technology . 


Limitations 

Due to the limitations of available data, it was 
difficult to assess the relative cost difference of the 
entire NCC curriculum, other than by specific 
courses as shown in this report. Likewise, retention 
data were defined by the completion rate rather than 
by other potential definitions. IUPUI is making an 
effort to track individual students through their 
curriculum more accurately so that these data may be 
more robust in the future. 

Given the different number of students taught in the 
traditional v. NCC classrooms (NCC classroom 
sections are limited to 22 students each while 
traditional sections are limited to 27 students each), it 
is difficult to make accurate cost comparisons 
between NCC and traditional instruction on a course 
section basis. Accordingly, the cost per student 
comparison is used. 

The assignment of faculty to course sections (NCC 
and traditional) has an important impact on cost since 
such a high percentage of total course cost (ranging 
from 62 percent to 90 percent) is attributable to 
personnel. Faculty exposure to and familiarity with 
instructing with computers can affect the effort 
required of them, and therefore cost as well. 

Further, the non-random, yet somewhat planned 
assignment of instructors to sections (with widely 
varying levels of pay and experience) can also 
impede the cost comparison effort. 

This course level comparison - which acknowledges 
that W131 NCC instructors were teaching fewer 
students - suggests that networked computer 
classroom instruction may result in reductions of 
instructor effort for some activities. Associate faculty 
reported spending, on the average, 22 percent fewer 
hours preparing for class, 13 percent fewer hours 
teaching the class, and 15 percent fewer hours 
assessing students’ work than their colleagues who 
taught traditional sections. These differences might 
well be attributed to a lighter teaching load in terms 
of student numbers. NCC instructors reported 
spending more time in office hours, advising, and 
orientation/workshops, but still reported devoting six 
percent fewer total hours to all of their W1 31 -related 
duties than did those assigned to traditional 
instruction. It must be noted that the accuracy of data 
regarding faculty effort devoted to each activity may 
be questioned since it is self-reported. 

Classroom conditions may be another factor 
confounding the calculation of successful course 
completion. The Mary Cable Building, where only 
traditional classroom instruction is available, is in 
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disrepair and so may affect students’ performance as 
well as retention. 

There may also be a confounding factor involving 
registration. NCC classroom sections are listed first 
in the schedule of classes. The potential for students 
to choose the sections they see first may have a self- 
selection impact on the types of students who 
ultimately enroll in NCC sections versus traditional 
sections, therefore affecting the amount of time 
faculty then spend on their classes. 

The registration factor may have a minimal effect on 
comparative section costs, but its impact on overall 
curriculum cost may not be trivial - it is believed that 
some students ignore their writing placement 
recommendations and enroll at course levels for 
which they are not prepared. If these students move 
to the appropriate course level after classes begin, the 
enrollment in the sections they left obviously will be 
reduced. The resulting reduction in a section 
enrollment may have a greater cost impact in the 
computer classroom section where enrollment is 
limited. But, the actual number of students who fall 
in this category is believed to be small and is not 
recorded. 

Observations 

The model gives an accurate picture (excluding the 
allocated Responsibility Center Management charges 
for the school and administrative overhead) of the 
current total costs of providing writing instruction 
with and without computer assistance. As is often the 
case in such studies, this picture may raise more 
questions than it answers. The application of the 
economic model to the core writing courses of the 
English department has generated much discussion 
and many questions. Some of the key issues raised 
follow. 

The English department determines which 
composition courses are to be offered by computer 
(NCC format). The decision to eliminate a course 
section is based on student demand for the 
curriculum and attempts to maximize the class 
section capacity. A typical NCC section enrolls a 
maximum of 22 students (23 students are allowed to 
enroll with the anticipation that at least one student 
will drop or withdraw). This maximum is due to the 
available number of desktop computers. Traditional 
classroom sections enroll approximately 27 students. 
If a course section is to be canceled due to low 
student enrollment, there is a conscious decision to 
eliminate a traditional classroom section before 
eliminating a NCC section. A NCC section has never 
been deleted from the schedule at IUPUI. 


The opportunity for a student to take a NCC course 
section increases as he/she progresses through or 
places into a higher level in the curriculum. The 
number of NCC sections in course W001, 
Fundamentals of English, was eight out of 61 
sections in 1994-95. For course W131 there were 57 
NCC sections out of 138 total sections offered. For 
course W231, Professional Writing Skills, there were 
32 NCC sections out of 43 sections offered (the only 
traditional sections were held off-campus). 

There is a higher TOTAL cost for W131 NCC course 
sections because these require contributions by 
technical staff, investments in hardware and software, 
as well as some supplies, that are not needed in 
traditional sections. Essentially, the NCC sections 
simply apply technology within the constraints of the 
traditional classroom process. For example, since the 
computer classroom instructors cannot assess 
students’ writing on-line or obtain access to student 
work from a remote site, they must incur costs in 
making paper copies in order to assess their students’ 
work and be on campus to do so (at least to receive 
the students’ work). This raised probably the most 
critical cost comparison issue within the study: 

whether the high differential costs were inevitable, 
or simply one of the results of not exploiting the 
strengths of technology in instruction by 
reengineering the instructional process. 

Another worksheet was created to display the per- 
student costs of traditional and classroom computer 
sections of the four core writing courses of W001, 
W131, W132, and W231. The entire curriculum 
appears to generate revenue for the department and 
school. The high cost course appears to be the NCC 
course W001 (a course cost figure was determined 
using the third-year associate stipend, with the 
average cost per section for computer cost 
contributions for W001 - since no surveys were 
returned for associate faculty in this course). This 
revenue-generating assessment is based on tuition 
revenue of $277 per student (three credit hour course 
@ $92.20 per credit hour). The low cost courses 
appear to be the traditional sections of the W001 and 
W131 courses. 
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Conclusions /Recommendations 

Although the application of the Economic Model to 
the English Department’s core writing curriculum did 
not accomplish as far reaching an analysis as had 
been considered originally, it did prove its ability to 
address cost issues relative to the application of 
technology to instruction. It should be noted that the 
extent of use of the model will be limited by two 
factors: a number of confounding variables that 
ideally would be controlled, and limitations of data 
available for analysis. 

As such, the model could be applied to a wider 
degree of analysis beyond the basic cost per student 
comparisons given the following recommendations 
for control of confounding variables and data 
collection: 

course 

❖ balanced opportunities for registration of 
students to NCC and traditional formats 

instructors 

❖ assignment of faculty controlling for differences 
in rank 

❖ assignment of faculty using controlled levels of 
experience 

❖ assignment of faculty using controlled degrees of 
NCC savvy 

❖ control of the accuracy of faculty reported effort 
to activity 


In summary, the use of technology-enhanced 
instruction makes a difference in cost outcomes, 
influenced by a number of factors. How these factors 
are managed will determine the most cost-effective 
means of instruction. Two key questions follow from 
this study: are we successfully exploiting technology 
to its fullest capabilities, and does the impact of 
technology warrant its investment cost? Both will 
require consideration of broader Flashlight Program 
findings before they can be answered satisfactorily. 


students 

❖ assessment of students in specified instructional 
formats 

❖ collection of data by sequenced courses 

❖ collection of thorough retention data 

❖ defined time period of retention 
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CASE STUDY 2: 


University of Pennsylvania 

Reducing the Costs of Laboratory Instruction through the Use of On-Line 

Laboratory Instruction 


David P. Pope and Helen L. Anderson 
School of Engineering and Applied Science 
University of Pennsylvania 


Abstract: This paper describes a new program for teaching undergraduate laboratories in the 
School of Engineering and Applied Science at the University of Pennsylvania, based on the idea 
that laboratories can be taught more efficiently, less expensively, and better with the use of World 
Wide Web-based technology. This technology is used to help the students prepare themselves 
before coming to the laboratory by becoming acquainted with the equipment, going through 
pre-lab exercises and taking pre-lab quizzes, both on the content of the work and on the safety 
considerations of the laboratory, all through web-based exercises. 

This project demonstrates that web-based teaching tools can both improve the quality of an 
undergraduate laboratory and, at the same time, reduce costs. The project team accomplished 
this by making a number of changes in the way laboratory courses are offered: 

• Students prepare for laboratory periods by accessing lab information on the web, 
beginning the laboratories online before class. 

• An institution-wide system of on-line grading reduces the time faculty must perform 
record-keeping activities, and gives students more accurate and timely feedback on 
performance. 

• Using software on desktop computers to convert the computers into “ virtual instruments ” 
dramatically reduces the cost of laboratory equipment maintenance and replacement. 

• Increasing the utilization of laboratories significantly reduces the cost of teaching 
laboratories by reducing the need to construct new laboratory facilities to fulfill 
laboratory curriculum requirements. This project increased the total usage of the 
Electrical Engineering laboratory through sharing laboratory modules on the web across 
engineering departments. 

The Flashlight (activity-based) approach to cost analysis shows that this project reduced the 
costs of teaching some labs by 30%. With web-based technology, we have improved the students ' 
learning through their pre-laboratory activities. This, in turn, has increased the substantive work 
that occurs in the actual laboratory session, while reducing the time requirements for the 
laboratory session. All lab sessions are real, hands-on experiences for students, not simulations. 
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Introduction 

Engineering and science curricula are extremely 
expensive to teach because they involve many 
specialized classroom and laboratory courses. Over 
the past few decades, partially in response to 
economic pressures, most engineering schools have 
come to rely more on lecture-format courses (with 
more analysis and theory) and less on laboratory- 
format courses and hands-on experiences. More 
recently the faculty members of the University of 
Pennsylvania School of Engineering and Applied 
Science (SEAS), as well as at many other engineering 
schools, have begun to re-emphasize laboratory 
studies, but the costs are still prohibitive. The 
challenge schools face is how to include high quality, 
hands-on laboratory courses in the curriculum while 
reducing their costs. Since the largest single cost in 
any educational institution is labor, decreasing costs 
requires reducing the time it takes to deliver the 
educational product. 

Through a project in the School of Engineering and 
Applied Science, funded by the Andrew W. Mellon 
Foundation, faculty from four Engineering 
departments. Bioengineering, Electrical Engineering, 
Materials Science, and Mechanical Engineering, 
explored how to use the new information 
technologies to make laboratory education less 
expensive and more effective. The goals of this 
program included: (1) increase the efficiency of 

using expensive laboratory facilities and instructor 
time; (2) reduce damage to expensive laboratory 
equipment; (3) increase the time faculty have 
available for personal interaction with students in the 
laboratory; (4) create an educational approach that is 
portable and can be readily adopted by other 
institutions; and (5) demonstrate through rigorous 
assessment that the project has indeed achieved its 
goals. 

The high costs of laboratory teaching arise 
principally from the costs of personnel, space, and 
equipment. It simply takes a great deal of time to 
ensure that experimental stations are maintained in 
proper working order. An even more important 
challenge for both faculty and students is that of 
instructing students at the start of any laboratory 
session in the safe and proper use of the equipment. 
The central focus of this project was on developing a 
way to deliver on-line, apparatus-specific instruction 
asynchronously along with a competence-check so 
that students can learn how to operate expensive 
equipment before coming to the laboratory and also 
rehearse aspects of the experiments they will carry 
out. 


The result of using this on-line pre-laboratory 
experience is that less time is required in the 
laboratory for each class, and therefore both 
personnel and capital costs of teaching each student 
are reduced, or conversely, more can be taught in 
each laboratory, so fewer labs are required for the 
total curriculum. Equally important, however, is the 
fact that the quality of the laboratory experience is 
significantly improved for the student. In addition, 
having users who are better informed results in less 
damage to expensive equipment and lower repair and 
replacement costs. Thus, making multi-media pre-lab 
modules available, complete with quizzes and 
tracking, reduces the personnel and capital costs per 
student while improving the learning environment. 

Another reason for the high personnel costs of 
laboratory experiences relates to the challenge of 
academic bookkeeping. For example, it is essential to 
track which students have completed what 
experiments, whether the resulting reports are graded, 
and what to do about team members who were absent 
on the day of the experiment. Instructional staff 
members usually deal with most of these questions 
manually, but it commonly happens that, after these 
routine issues are handled, too little time is left for 
substantive exchanges between instructor and student 
about content. As part of this program, SEAS 
purchased software that includes a grading 
spreadsheet on which students can electronically 
access only their own individual records and thereby 
eliminate much of the routine paperwork. This then 
allows more time for the kinds of substantive 
discussion that both students and faculty prefer. 

Finally, a comment on the general nature of the 
problem itself: the best reason for performing a cost 
analysis like this is to force oneself to truly 
understand what actually controls the costs of an 
academic endeavor. Academics tend to ignore the 
costs of teaching a course, simply because the rigors 
of the course itself are enough to occupy us. When 
SEAS first started this exercise at Penn, faculty did 
not have a clear idea of the costs involved until later 
in the program when the project team used the 
Flashlight methodology. Using Flashlight’s approach 
to the modeling of costs helped the team track down 
the hidden as well as the more obvious expenses of 
running the laboratories. 

Briefly, the team concluded that, because the main 
costs involved in teaching an undergraduate 
engineering laboratory are personnel, space, and 
equipment, then cost reductions must come in those 
areas. Reducing the costs of other elements, which 
contribute only a small part to the total costs of 
running the lab, does not significantly cut the total 
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cost of the lab. Therefore, the project team focused its 
efforts on major costs. However, given the nature of 
our university, the team had no intention of reducing 
the amount of “face time” our undergraduates have 
with faculty. To reduce personnel costs while 
remaining true to our principles, the project had to 
reduce the time the faculty and staff previously had 
been compelled to spend on less valuable activities, 
like handling paper, academic bookkeeping, and 
competence checks. On-line pre-lab instruction gives 
students the opportunity to prepare for lab, including 
the handling of expensive equipment, rehearsing the 
lab experiment, and taking a pre-lab competence 
check. Consequently, students are prepared to begin 
the experiment immediately at the start of the lab 
session, reducing the time requirements for a typical 
lab session from three hours to two. 

More challenging was the puzzle of how to reduce 
the costs of space and equipment. One key insight 
was to realize that at Penn, as at many other 
institutions, laboratory space and equipment lay idle 
much of the time. The key lay in increasing 
laboratory utilization while streamlining laboratory 
student/faculty “face time” requirements by 
concentrating “face time” on substantive work. The 
true costs of the space and equipment can only be 
reduced through greater utilization of the space and 
equipment, i.e., by having more courses and more 
students use the facility per semester. Since many 
laboratories are highly specialized, greater utilization 
is difficult - unless faculty members are willing to 
collaborate on developing shared space and 
equipment between departments. This requires a 
cultural change in the faculty, a much more difficult 
type of change within an institution, without which 
cost reductions of this nature are impossible. 


The Studies 

Software Evaluation 

One of the team’s first efforts was directed at trying 
to develop an in-house quizzing program customized 
to our needs for pre-laboratory instruction. However, 
we soon switched to commercial software packages 
because it became clear that they are much less 
expensive to establish and to maintain. During 1999, 
the project team used both Mallard™ and Blackboard 
Courselnfo™. We discovered that these packages 
saved us not only programming time but also the 
whole process of negotiating and prioritizing the 
precise needs of faculty members across the school. 
Packages either provided the services we wanted or 
they did not. If they did not, we simply considered 


other packages. We still needed to connect the 
packages to our local infrastructure, such as pre- 
loading enrollment information, but we would have 
done that whether we had used a custom package or a 
commercial package. 

In the sophomore Electrical Circuits and Systems I 
course, pre-lab material was posted on the web in fall 
1999. We measured preparation through 13 weekly 
quizzes delivered with Blackboard Courselnfo. We 
surveyed the students about their preparation, and 
found that 92% agreed or strongly agreed with the 
statement, “I am better prepared for my labs because 
I use Blackboard.” One student commented, “I 
probably would not have done the pre-labs if there 
was not a pre-lab quiz.” Another said, “It allows me 
to see what I've done wrong, which gives me the 
chance to look through my initial calculations to find 
my mistake.” Yet another student said, “By using the 
Blackboard system to take the quiz, we don’t have to 
waste time in class taking the quiz.” Many students 
commented on the immediacy of the results, such as, 
“It is very helpful because we get back our results 
immediately.” 

In Electrical Circuits and Systems II, after a written 
pre-lab assignment was replaced by an online quiz, 
the Undergraduate Lab Manager reported “no 
significant difference” between the actual lab 
performance of students who took the quiz and 
students the previous semester who did written 
assignments. However, he said, “I could immediately 
get feedback. Within two minutes it would tell me 
how well written our labs were and how well 
prepared our students were.” 

Though Mallard has been a very effective teaching 
tool in Electrical Engineering, it is difficult for 
instructional staff to learn. Quiz development 
requires effort from student workers or support staff. 
On the other hand, Blackboard has less quizzing 
functionality, but it is much easier to learn. Because 
of this project in the School of Engineering and 
Applied Science as well as in the Mellon Writing 
Project in the School of Arts and Sciences, both 
funded by the Mellon Foundation, and because of a 
University committee called “New Tools for 
Teaching,” Blackboard software has been widely 
adopted throughout Penn. Approximately 1500 
students used Blackboard in the fall of 1999. Usage 
has expanded each semester, and in fall 2001, over 
16,000 students in 1400 courses across seven schools 
at Penn used Blackboard Courselnfo. The largest 
single course using the program is ECON 1, with 
enrollment of 900 people. 

Based on what the team learned at The TLT Group’s 
Flashlight workshop, “Evaluating Web-based 
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Based on what the team learned at The TLT Group’s 
Flashlight workshop, “Evaluating Web-based 
Courses,” we did a survey of students about their 
experiences with Blackboard Courselnfo. We looked 
for changes in teaching methods made possible at a 
reasonable cost by the software, and found several. 
Over 400 students replied to the survey in December 
1999. Students reported more prompt feedback, time 
on task, appeal to different learning styles, active 
learning and collaborative learning. Through the New 
Tools for Teaching committee, these results were 
used as publicity to expand usage of Blackboard. 

Specifics of the Effort in an Electrical 
Engineering Course 

On-Line grading systems 

Prior to the changes made in laboratory instruction, 
students arrived at their laboratory sessions with 
varying degrees of preparedness. On a typical day in 
a typical lab, approximately 1/3 of the students were 
fully prepared to do the planned lab experiment, 1/3 
were only somewhat prepared, and the remaining 1/3 
were not prepared at all. Now, with the pre-lab 
instruction on the web, instructors are able to 
motivate students to invest in laboratory preparation 
by including a pre-lab quiz with the web instruction. 
Previously, instructional staff manually graded 
pre-lab assignments and quizzes and returned them to 
students the following week after the laboratory 
experiment was completed, when the feedback was 
no longer useful. In fact, we have found that, in 
general, a remarkably large amount of time was spent 
in communicating grades to students, checking to see 
if grades have been changed after exams are re- 
graded, and in checking the overall accuracy of the 
grades. By putting all lab and course grades on-line 
so that the students can see only their own grades, the 
time spent in these activities has been greatly 
reduced, and furthermore, the students greatly 
appreciate the access. (We have found, for example, 
that they catch a huge number of grading and 
recording mistakes, most of which were probably 
missed in the past.) After experimenting with a 
system developed here at Penn, we finally adopted 
the Blackboard system. Everyone who has used it is 
very pleased with it. 

Laboratory Equipment 

Rendering desktop computers in the laboratory into 
“virtual instruments” has the greatest potential for 
future cost savings in this category. Through the use 
of special software, the computer can be converted 


into the instrument of choice. For example, the 
computer could be an oscilloscope, or a digital 
voltage meter, or a constant voltage power supply, or 
all three simultaneously. Since every laboratory is 
now equipped with desktop computers, the only 
additional requirement is the software to create the 
virtual instruments. To do this, we purchased a site 
license for Lab VIEW software for the entire School 
of Engineering, and saved $20,000 for the School, 
simply because some individual laboratories were 
already using the software, and the total software 
expenditure prior to the site license exceeded the cost 
of the site license by the above amount. The cost of 
the site license is divided among all classes that use 
LabView. This is reflected in Table 2-1. However, 
the $20,000 savings to Penn from the purchase of the 
LabView site license is additional to the savings in 
Tables 2-1 and 2-2. Now, as more and more students 
and faculty become familiar with the software, we are 
finding that we need not purchase new stand-alone 
instruments with the same frequency as before and 
therefore we are saving substantial amounts on 
equipment purchases. See Table 2-1 for equipment 
savings in the web-assisted lab vs. the traditional lab. 

Activity-Based Cost Analysis 

We have analyzed the total costs of teaching our 
laboratories using the activity-based cost method in 
an attempt to see if we are, indeed, saving money 
with the web-based laboratory instruction, and from 
which specific elements those savings come. We 
approached it by asking the following specific 
question: what is the cost difference between a 
traditional section of Engineering Laboratory, 
EE 205, which meets once per week for 15 weeks, 
and a web-assisted section that meets with the same 
frequency for shorter times but which makes use of 
web-based tools for teaching, quizzing, instrument 
simulation, and data analysis? 

We approached it based on the assumptions outlined 
in Table 2-1 and then broke the costs down using the 
methods from version 1.0 of the Flashlight 
Handbook, and using the following assumptions 
when entering the data into the spreadsheets: 

1. Salaries are averages for the School of 
Engineering for the particular category. 

2. Hours are estimates, based on 
discussions with the faculty. 
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3. 


Space cost is the average paid by the 
School of Engineering to the university 
for the laboratory space we occupy. 

4. By ensuring that the students are well 
prepared to do the laboratory exercises, 
based on web-based exercises and web- 


based quizzes performed prior to class, 
the total time for the lab was reduced 
from three hours to two hours. 

5. Most laboratories now contain personal 
computers already, so no additional 
costs are shown for PCs. 


TABLE 2-1 

Assumptions of the cost analysis 

Comparison of course elements for EE 205 in traditional lab vs. web-assisted lab 


Traditional Laboratory 

Web-Assisted Laboratory 

Printed syllabus & laboratory manual 

Web-based syllabus & laboratory manual 

Taught in conventional laboratory, 3 hours per week 

Taught in "augmented conventional” lab, 
2 hours per week 

Serves 30 students 

Same 

Laboratory space = 1 ,000 sq. ft. 

Same 

Must purchase 15 sets of (2 students/set): 
+ PC - high end, $1,000 
+ Oscilloscope, $2,100 
+ Function Generator, $1,400 
+ Digital Multimeter, $1,000 
+ Programmable Power Supply, $1,000 
Total cost per 2 student/set = $6,500. 

Must purchase 15 sets of (2 students/set): 
+ PC - high end, $1,000 
+ Programmable Power Supply, $1,000 
+ Interface Board, $500 
Total cost per 2 student/set = $2,500. 

Per class charges 
+ Lab View Software, $250 
+ Increased server capacity, $200 
+ Blackboard site license, $250 
Total per class charges = $700. 

Grades delivered in traditional way, i.e. w/o much 
comparison with others 

On-line grades w/fiill statistical comparisons 

Data handled in traditional way, either hard copy 
(usually) or floppy. 

Data, both numbers & images distributed on net 

Requires 1 instructor, 1 technician, & grader 

Same (plus additional IT help) 

Quizzes (if at all) with paper & pencil. 

Quizzes on-line by Blackboard and cover safety, 
equipment use as well as content 

Reports & iterations are hard copy 

Final report is hard copy, iterations are on-line 


55 












TABLE 2-2 

Comparison of cost to teach 30 students in EE 205 in a traditional lab vs. a web-assisted lab 


Category 

Traditional Lab 

Web- Assisted Lab 

% Difference 
Traditional vs. Web 

Faculty 

$11,781 

$8,709 

-26% 

Lab Coordinator 

1,559 

2,095 

+34% 

Teaching Asst. /Grader 

2,132 

1,451 

-31% 

Staff Asst./CETS 


139 

+100% 

Total Personnel Cost 

$15,472 

$12,394 

-19.8% 

Printing 

1,200 


-100% 

Space 

1,875 

1,250 

-33.3% 

Equipment/Software 7 
year depreciation 

5,571 

3,100* 

-44.3% 

Total Lab Cost 

$8,646 

$ 4,350 

-49.6% 

Total Cost per 30 student 
section 

$24,118 

$16,744 

-30.5% 


^Transition to virtual instrumentation is not yet complete. 


The spreadsheets for the traditional approach to teaching the laboratory and the web- assisted approach are shown in 
Tables 2-3 and 2-4. 

Note that the web-assisted course costs over $7000 less to teach— but there is no one item that accounts for most of 
the savings. The savings come from less faculty time in the classroom and less time spent on the details of testing, 
grading and reporting. The other large savings come from lower expenditures for equipment and software in the 
web-assisted course. The savings in faculty expense may be more ephemeral than real, however. Since faculty teach 
whole courses, not fractions, the savings shown here may not be realized, unless the faculty can be convinced that 
this course really takes less effort now, and therefore that faculty member will take on some other additional 
function for the department, which faculty have done at Penn. 

Thus, the savings can be quite substantial, over 30% in this case. But more importantly, we have increased the 
technology in the course and decreased the cost. This is contrary to the normal trends in our institution. 
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TABLE 2-3 

Spreadsheet for traditional laboratory 


99091 7 Copyright © 1 999 Indiana University. /VII Rghts Reserved. 


Economic Model, comparative exist matrix 

205 -Traditional 
30 students, 2 sections 


actiwty task 

person/ unit 
/erg. 

total oost 

Faculty 

Technician 

GraderfTA) 

Staff 

Asst (computing) 

Printing 

Space 

1 $/hour 

57.75 

$/hour 

27.84 

$/ hour 


12.62 

$/hour 


27.84 

cost 

cost 

| hours | cost 

hours | 

cost 

hours | 

1 

cost 

hours 

J _ 

cost 

| Develop Syllabus & Lab. Manual 

2,125 

24.0 $ 

1,386 

5!oT 

139 

- 

$ 

- 

- 

$ 

- 


600 

A 

- 1 

Revise course, syllabus 

924 

16.0 

924 


- 



- 

- 


- 


- 


- 

Revise manual, tutorials 

601 

8.0 

462 

5.0 

139 



- 



- 





Print manual and syllabus 

600 


" 


‘ 



■ 



' 


600 



| Develop Equipment & Software Needs 

86 

1.0 $ 

58 

1.0 $ 

28 

-$ 



$ 

600 

$ 

_ I 

Select New Instruments 

I 86 

1.0 

58 

1.0 

28 

- 


- 



- 





Evaluate Computer Needs 

0 















|Pirchase& install new equi pi & software 

6,295 

0.0 $ 

- 

26.0 $ 

724 

- 

$ 

- 

- 

$ 

- 

L$ :_J 

L§ 

Purchase equipment & supplies 

5,850 


- 

10.0 

278 



- 



- 




- 

Install new equipment & train tech. 

223 


- 

8.0 

223 



- 



- 





Test new equipment 

223 



8.0 

223 











|Train Personnel 

0 

0.0 $ 

- 

0.0 $ 

- 

$ 

- 

- $ 


- 

A 

- 1 

Train faculty 

0 

■ 

■ 

' 




| Teach the corse 

8,039 

79.0 $ 

4,562 

24.0 $ 

668 

74.0 

$ 

934 

- 

$ 

- 

$ 

- 

$ 

1,875 | 

Training of students 

393 

4.0 

231 

4.0 

111 

4.0 


50 



- 





Oversee activity in lab 

6,654 

60.0 

3,465 

20.0 

557 

60.0 


757 



- 




1,875 

Office hrs, 12 hr/wk, 1 Fac+1 TA 

992 

15.0 

866 



10.0 


126 








| Testing, gracing reporting 

6,973 

100.0 $ 

5,775 

0.0 $ 

- 

95.0 

$ 

1,198 

:_J 

LI i_J 

A _J 

Prelab Quiz and exercise prep 

641 

10.0 

578 


- 

5.0 


63 



- 





Grading Prelab Quiz and Exercise 

2,111 

30.0 

1,733 


- 

30.0 


378 



- 





Grading laboratory reports 

4,222 

60.0 

3,465 


■ 

60.0 


757 



■ 






$ 23,519 

204.0 | $ 

11,781 

56.0 | $ 

1,559 

I 169.0 | $ 

2,132 

o 

© 

m 

[T 

1,200 

[T 

1,875 | 
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TABLE 2-4 

Spreadsheet for web-assisted laboratory 

Economic Model, comparative cost matrix 


EE 205 - WEB Assisted 
30 students, 2 sections 




person / 












~ 


Equipment 8i 



unit/org. 

| Faculty 

| Technician (Sid) 

Grader(TA) 

Staff Asst-(CETS) 

Printing 

Space 


Software 




$/hour 

57.75 

$/hour 


27.84 

$/hour 


12.62 

$/hour 

27.84 






activity task 

total cost 

hours | 

cost 

hours | 


cost 

hours 


cost 

hours | 

cost 

cost 


cost 


cost 

| Develop Syllabus & Lab. Manual 

% 

1,870 

29.0 $ 

1,675 

7.0 

$ 

195 

- * - 

- * - 

$ - 

s - 

i n 

Revise course content, syllabus 


924 

16.0 

924 



. 



. 


. 



. 


. 

Revise manual, tutorials 


601 

8.0 

462 

5.0 


139 





. 





. 

Maintain Web site 


344 

0 

5.0 

289 

2.0 


56 



- 


- 






| Develop Equipment & Software Needs 

$ 

931 

6.0 $ 

347 

21.0 

$ 

585 

- $ - 

- 5 - 

$ 

* ■ 

i n 

Select New Instruments 


86 

1.0 

58 

1.0 


28 

. 


- 


. 





. 

Evaluate Computer Needs 


139 


. 

5.0 


139 



. 


. 





. 

Evaluate Software Needs 


706 

5.0 

289 

15.0 


418 



- 


* 





- 

|purchase & install new equip. & software 

$ 

4,242 

0.0 - 

36.0 

$ 

1,002 

... - $ - 

5.0 $ 

139 

$ 

s - 

$ 

3,100 

Purchase equipment & supplies 


2,678 

0.0 

. 

10.0 


278 



. 


. 





2,400 

Purchase & update software 


918 


- 

10.0 


278 




5.0 

139 





500 

Additional server costs 


200 


. 













200 

Install new equip, soft. & train tech. 


223 


. 

8.0 


223 





. 






Test new equipment 


223 


- 

8.0 


223 










- 

|Train Personnel 

$ 

0.0 $ 

0.0 

$ 

- 

$ - 

$ 

$ - 

$ - 

$ 

Train faculty 

0 

- 






- 

|Teach the course 

$ 

4,020 

45.8 $ 

2,645 

11.3 

$ 

313 

50.0 

S 

631 

$ 

$ - 

$ 

1,250 

$ 

Training of students 


154 

1.5 

87 

1.5 


42 

2.0 


25 







. 

Oversee activity in lab 


3,126 

22.1 

1,276 

9.8 


271 

26.0 


328 





1,250 


. 

On line support for course 



12.0 





10.0 










Office hours 


740 

10.2 

589 



* 

12.0 


151 







- 

besting, grading, reporting 

$ 

4,799 

70.0 $ 

4,043 

0.0 

$ 


65.0 

$ 

820 

- 5 ■ 

$ 

$ - 

i n 

Prelab Quiz and Exercises 


578 

10.0 

578 




5.0 


63 







. 

Grading Prelab Quiz and Exercises 


0 

0.0 

. 






_ 







. 

Grading laboratory reports 


4,222 

60.0 

3,465 




60.0 


757 







* 



15,862 

150.8 |$ 

8,709 

75.3 | $ 

2,095 

115.0 

i. 

1,451 

5.0 | $ 

139 

$ - 

$ 

1,250 

$ 

3,100 
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A Reality Check 

The savings realized in the case of the EE laboratory 
are real in the sense that by making the changes we 
can utilize the same laboratory for more courses, 
reducing the demand for new laboratory facilities. 
We could do this because the laboratory was already 
almost 100% utilized, and we needed more space for 
other laboratories that could use the same space and 
equipment. By making the changes in this course, we 
could accommodate another course in the same 
laboratory and therefore avoid building a new one. 

This is an unusual situation, at least in our institution. 
Most teaching laboratories are not used 100% of the 
time. In fact the usage is much less than that. If we 
are to reduce costs, then we can only do it by sharing 
such laboratories across departments, i.e., by actually 
reducing total laboratory space and using it more 
efficiently. This is not as radical an idea as it seems, 
because these new laboratories could be better 
equipped, staffed by highly skilled technicians, and 
still cost less than they do now. In fact, one of the 
reasons why we could reduce the costs of the 
Electrical Engineering laboratory described above is 
because this laboratory is used for Electrical 
Engineering, Systems Engineering and Computer 
Science courses. Were it only used by the Electrical 
Engineering Department, no cost savings would be 
possible. 


An Unexpected Bonus of this Effort 

As the group of faculty and staff involved in this 
effort discussed the laboratory experiences in the four 
different departments, we discovered a great overlap 
in the experiments used by the different departments, 
i.e., the number of distinct experiments is limited and 
is far less than the total number of experiments 
performed in our undergraduate labs. For example, 
mechanical testing of materials is done in at least four 
different departments, using very similar techniques: 
in Civil Engineering, Mechanical Engineering, 
Bioengineering, and Materials Science. The only real 
differences are the gripping methods (tension vs. 
compression) and the materials themselves (concrete, 
steel, chicken bones or differently heat-treated 
A1 alloys). 

The faculty decided that it makes much more sense to 
concentrate these kinds of activities in one 
departmental laboratory under the supervision of one 
technician. As a result, students from all these 
departments are performing mechanical tests in the 
Materials Science laboratory, but as part of courses 
offered in the students’ home department. The cost 


savings associated with this approach are obvious, 
but the approach also offers additional advantages. 

Many times it is desirable to offer laboratory 
experiences as part of a regular, non-laboratory 
course. Normally, however, the activation barrier for 
doing this, in terms of obtaining test samples, 
preparing the equipment, and training the personnel 
is insurmountable, so it is not done. Now that we 
have many of these experiments on the web, and 
since there are personnel trained to do these 
experiments, we have the option of “Lab a la Carte,” 
i.e., an instructor can go to the web, pick and choose 
from the experiments already being performed as 
parts of other classes (remembering that the number 
of distinct experiments we do is very limited) then 
schedule the experiments. 

Persuading faculty to share laboratory space and 
experiments requires convincing them that they will 
realize a net gain for their students. These gains can 
be demonstrated only by providing faculty with better 
labs, better equipment, and better support with more 
highly skilled laboratory technicians. 

Conclusions 

The project team has shown that by using web-based 
teaching tools we can both improve the quality of an 
undergraduate laboratory while, at the same time, 
reduce costs. The team accomplished this by making 
a number of changes in the way laboratory courses 
are offered: 

1 . Now students prepare for laboratory periods 
by accessing more information on the web 
and beginning the laboratories online before 
class. Students can get a real feel for the 
experiments before coming to the laboratory, 
and furthermore, faculty can convey essential 
safety information (and give quizzes to see 
that they actually know the information) prior 
to the start of the laboratory period. This 
preparation has a significant impact on 
student productivity. 

2. An institution- wide system of on-line grading 
greatly increases the efficiency and accuracy 
of the grade-reporting process. In the pre-lab 
quizzes, students receive immediate 
feedback, because of automatic grading. 
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3. Using special software on desktop computers 
to convert the computers into “virtual 
instruments” can dramatically reduce the 
costs of laboratory equipment. Since every 
laboratory station in most teaching 
laboratories now is equipped with a 
computer, there is no additional cost 
associated with the computers. Faculty 
adopted Lab View as our standard, and 
purchased a site license for the School of 
Engineering and Applied Science. 

4. Using the Flashlight (activity-based) 
approach to cost analysis, we have estimated 
the costs of teaching some of our 
laboratories. Improved student preparation 
allows us to decrease laboratory periods from 
three hours down to two hours, resulting in 
substantial savings in space and personnel 
costs (and opening up the laboratory for other 
classes for which the construction of an 
additional laboratory space would otherwise 
be required.). These cost savings, combined 
with the savings associated with the use of 
Lab View software to replace hard- wired 
instruments, can exceed 30%. While these 
savings are not huge, they do constitute an 
important breakthrough because the costs of 
laboratory teaching have been increasing so 
rapidly for so long. 

5. The increased utilization of some of our labs 
through this project reduces the pressure on 
the school to build new labs to accommodate 
growth in the demand for hands-on 
laboratory experimentation for students. 
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Abstract: Costs of Rochester Institute of Technology s (RIT) site-based and anytime/anywhere 
(asynchronous) courses were compared to their on-campus counterparts in this preliminary 
study. Our main objective was to use the model to obtain a detailed, activity-based perspective 
on the operational costs of delivering distance learning courses. Results are being used as part 
of the RIT Distance Learning Strategic Plan. 

RIT was the second school to use the beta version of the activity-based cost analysis model 
developed at Indiana University Purdue University-Indianapolis (IUPUI) for the Teaching, 
Learning and Technology (TLT) Group Flashlight Program. RIT modified the model to fit its 
business objectives with activities categorized into six course components: faculty preparation, 
presentation, interaction, assessment, practice/application and evaluation. Within each 
component, tasks were identified and a survey instrument was developed to gather data about 
time on task, direct costs, and indirect costs for each activity. 

Eight courses were selected from a possible 148, representing three colleges, undergraduate 
and graduate levels. Courses in the two primary formats of distance learning were identified as 
site-based and anytime/anywhere. Faculty members were selected for their experience in 
distance teaching in order to focus on the costs of delivery rather than up-front development. 
Faculty selected taught both the on-campus and distance learning versions of the courses in the 
quarter in which the survey was conducted. 

Our findings reflected the wide variability in the activities involved in instruction. Faculty time 
was the largest and most variable portion of the total cost. The faculty members reports of 
teaching time per student varied by as much as a factor of three across the on-campus courses 
and by a factor of four across the distance learning courses. 

One surprising finding was the perception of time by the faculty. Comparing different formats of 
the same course, all faculty felt they spent more time teaching the distance learning versions of 
their courses. The totals of the self-reported hours, however, indicated that only three of the 
nine distance learning sections consumed more time per student. Five sections consumed about 
the same amount of time per student and one section consumed fewer hours per student in the 
distance learning version. 

The average cost of the anytime/anywhere courses and the site-based courses was roughly the 
same. The site-based courses had higher average costs supporting interaction and assessment. 
Anytime/Anywhere courses had higher average costs supporting preparation and practice. 
Comparing these costs to the average cost per credit hour of on-campus courses from the same 
college did provide some point of comparison. Results indicated eight of the nine course 
sections in our sample cost the same or less per credit hour as their on-campus counterparts. 
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Through this study, we gained a clearer understanding of our operating costs. We used the 
on-campus version of the course as a benchmark. Further study is needed to draw conclusions 
about the technology costs of the anytime/anywhere format, and the impact on faculty time of 
different technology choices. 


Introduction 

A 20-year veteran in distance education, RIT has 
offered distance learning courses in various formats 
since 1979 and introduced full-degree programs 
through distance learning in 1990 when it received a 
“New Pathways” grant from the Annenberg/CPB 
Foundation. Currently, RIT offers 20 of its degree 
and certificate programs in a variety of distance 
learning formats including synchronous and 
asynchronous forms. 

RIT is in the first phase of a six-year initiative to 
triple distance learning enrollment from 3.7% to 10% 
of total credit hours generated by the university. The 
Distance Learning Task Force Report indicated that 
RIT can reach its goal of 10% of credit hours 
generated within six years, assuming four critical 
conditions are in place. Those are a focused and 
committed marketing effort; serious commitment 
from the colleges and faculty to produce and offer 
academic programs and courses of the highest 
quality; an institutionally supported faculty 
development effort; and a supporting infrastructure of 
staff, equipment, software and training. 

The results of this cost analysis were intended to be 
used to address the following business objectives: 

• Identify all costs associated with teaching a 
variety of distance learning courses. This 
information will help determine faculty 
compensation for teaching and developing 
distance-learning courses, as well as help 
determine staffing and other resources. 

• Compare the costs of anytime/anywhere and site- 
based formats. Detailed costs can aid the 
transition of the college’s current site-based 
degree program to a site-independent format. 

• Determine costs associated with distance 
learning staff time and services in order to 
develop a “price sheet” for departmental charge 
backs. 

• Determine costs associated with teaching on- 
campus versions of the selected courses. Provide 
detailed information for comparing on-campus 
and distance learning courses that go beyond the 
current Institute-wide averages. 


• Identify revenue benefits of distance learning 
courses. 

Determining the Sample Courses 

Selecting courses for the study 

Eight courses from a possible 148 were selected as 
the target group (Table 1). We kept the sample small 
because we were beta testing the model. Courses 
were well-established graduate and undergraduate 
courses where the faculty member would not be 
developing a new course for the format. Courses 
were selected from both the synchronous and 
asynchronous forms of existing distance learning 
courses. Courses selected also had on-campus 
versions taught in the same quarter by the same 
instructor. One course was taught in three different 
formats in the same quarter, making nine sections of 
eight courses in the sample. 

Selecting faculty for the study 

The selected faculty members were not new to the 
distance learning format. Our purpose was to 
compare the costs of delivering course work rather 
than the costs of up-front development time and 
faculty training. Another advantage of having 
experienced faculty participating in this project was 
that they could better delineate tasks and recall what 
it takes to be thorough and complete. 

Faculty taught their courses in both a distance- 
learning and on-campus format during the quarter the 
study took place. The project team believed it was 
important to conduct the survey in the quarter that the 
courses selected were being taught so that the 
information would be as accurate as possible. 

This was our first cost study and the data we were 
gathering was personal to faculty (i.e. how they 
spend their teaching time), therefore we selected 
faculty who had a record of being supportive of 
distance learning projects. The selected faculty 
members are considered among the best RIT faculty 
and are proponents of distance learning within their 
departments. 
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Selecting distance learning and support staff 
for the study 

Staff consisted of a course developer, operations 
manager, project coordinators, faculty associates, and 
technical and clerical staff. This group interacted 
with faculty by coordinating schedules for recitations, 
phone conferences, and course materials. They also 
distributed materials, tests, quizzes and returned 


student work; helped to set up and administer online 
conferences; and designed evaluations for courses. 

Other participants included media production staff 
members who developed and produced the various 
media used (video and audio tapes, web pages, print 
materials, etc.). Support personnel within each 
faculty member’s department were also included in 
the study. 


Table 3-1: Sample courses 


College 

Lower 
Division 
100-200 level 

Upper 
Division 
300-500 level 

Graduate 

Level 

Major 

Formats 

Applied 
Science and 
Technology 


X 


Electrical 

Engineering 

Technology 

Site 

On-Campus 

Applied 
Science and 
Technology 


X 


Mechanical 

Engineering 

Technology 

Site 

On-Campus 

Applied 
Science and 
Technology 

X 



Environmental 

Management 

Anytime/ Anywhere 
On-Campus 

Applied 
Science and 
Technology 



X 

Information 

Technology 

Anytime/ Anywhere 
On-Campus 

Science 

X 



Mathematics 

Site, KEY 
On-Campus 

Science 

X 



Mathematics 

RAITN 

On-Campus 

Liberal Arts 


X 


Economics 

KEY 

On-Campus 

Liberal Arts 


X 


Political 

Science 

Anytime/Anywhere 

On-Campus 
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Building the Model 

We evaluated the costs of three types of instructional 
outputs in this study: courses taught using traditional 
means (on-campus courses); courses taught using 
technological means in an anytime/anywhere format; 
and courses taught using technological means in a 
site-based format. . For each type of output, we 
determined the activities and tasks that drive costs 
and developed activity grids (data worksheets). To 
determine die cost of each activity, we gathered data 
on direct and indirect costs including salaries, 
facilities, equipment, supplies, and services. We also 
surveyed faculty and staff to determine time spent on 
each task. 

A. Outputs 

On-campus - traditional classroom instruction. 

Anytime/anywhere - asynchronous instruction using 
a range of technologies including FirstClass 
conferencing, telephone conferences, web resources, 
videotape, audio tape, and CD-ROM. Faculty use any 
combination of these technologies. 


Site-based - RIT has three site-based programs: 

1. Electrical/Mechanical Engineering Technology 
Degree (E/MET)- Students meet in groups at 
community colleges and corporate sites. Courses 
use the Optel Telewriter™ (audio and graphics) 
for live interaction; videotapes and print 
materials for presentation are used 
asynchronously. Faculty members visit the sites 
several times per quarter. Testing is done with 
proctors at the sites. Some faculty use e-mail to 
support interaction. 

2. KEY Program (K-12) - A high school program 
that uses the Optel Telewriter (audio and 
graphics) for live interaction. Students also use 
videotapes and print materials for presentations 
in a class setting. Faculty visits the schools 
several times per semester and students come to 
RIT once per semester. Students meet in 
regularly scheduled, high school classes. Some 
asynchronous technology is used. Students take 
RIT courses for both high school and college 
credit. 

3. RAITN (K-12) - (Rochester Area Interactive 
Telecommunications Network) - A high school 
program that uses live, two-way audio and video 
for class-to-class instruction to sites. FirstClass is 
used to distribute print materials and to support 


some asynchronous communication outside of 
class time. Students take RIT courses for both 
high school and college credit. 

B. Activities and tasks 

The IUPUI model identified eight major activities for 
administering a course or program: 

• Concept/Projections 

• Prepare Course Content 

• Prepare Instructors 

• Prepare Materials 

• Market 

• Enroll 

• Instruct 

• Evaluate 

To meet our objectives, we narrowed our focus to the 
operational cost of instruction only and used six 
major activities as our primary course components. 
The six primary activity categories are: 

• Preparation 

• Presentation 

• Interaction 

• Assessment 

• Practice/ Application 

• Evaluation 

Within the primary activity categories, we identified 
approximately ten tasks. The tasks were the steps 
taken by faculty, Distance Learning staff, 
Educational Technology Center staff, and other 
support staff to teach a course. Here is one example: 

Activity: Presentation 

Tasks: 

Develop details of course content 
Plan Lectures 

Identify & Gather Existing Materials (text, 
readings, media) 

Determine Media Format 
Schedule/Obtain Production Resources 
Produce Own Materials 
Produce Materials with ETC 

Create Production Material 
Caption/Transcribe 
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C. Costs 

Costs associated with each task were identified and 
data sought for the activity grids through the 
university accounting office and distance learning 
records. Most costs associated with distance learning 
were available through Educational Technology 
Center and Distance Learning departmental records. 
Although tasks were identified for the classroom 
format, cost data were not available at the task level. 
Activity-based data was not available through our 
university accounting. 

Anytime/anywhere task costs include: 

• Media production (student time, staff time, 
facilities time, materials, captioning) 

• Faculty compensation for up-front development 
(not included) 

• Media duplication 

• Media licensing 

• Media distribution costs (scanning, file 

processing) 

• Postage 

• Transcription of non-captioned media and 
audioconferences 

• Telephone charges for audioconferences 

• Telecommunications line charges 

• Student audio bridge operators 

• Audio cassette supplies and postage 

• Orientation mailings (how-to information for 

students) 

• Mailing and faxing costs to exam proctors 

• Marketing costs (not included) 

• Advising (not included) 

• Academic program development (not included) 

• Service desk costs 

• Technical support costs 

Additional site format task costs include: 

Telecommunications line charges at sites 
Site visits 

RIT and site conferencing equipment 

Facility costs 

Proctor fees 

Site coordinator fees 


Classroom format task costs include: 

(With the exception of facility costs per square foot, 
costs for the identified tasks were not available at the 
detail needed for this study.) 

Room (cost per square foot for facility) 

Installed equipment 

AV service 

Media support 

Library 

Copying 

Sign language interpreters and note takers 
Faculty costs 

In order to isolate delivery costs, we chose to 
normalize faculty costs so that our results would not 
reflect variability in salaries and the differences in 
adjunct and full-time compensation. We used RTFs 
average salary and benefits for full-time faculty to 
make the hourly cost the same in each course. Also 
included in faculty hourly cost was an estimated 
average for office cost per square foot, computer, 
furniture, supplies, telecommunications, and 
overhead. 

Staff costs 

Detailed information was used to populate the hourly 
cost worksheets for distance learning staff. Data 
included individual salary and benefits, office cost 
per square foot, furniture, supplies, equipment, 
telecommunications, and overhead. 


Building costs 

Data for the building cost worksheets for the 
classrooms, distance learning offices, and production 
facilities were obtained from university accounting 
and facilities. 


D. Surveying time-on-task 

The most complex part of the study was surveying 
faculty and staff to determine hours spent on each 
task. The survey matrix evolved from a written, 
essay-style survey form to a more complex and 
detailed interview grid. The team decided that with 
the small sample group, interviews rather than a 
stand-alone survey would be the most effective way 
to gather time information. Interviews would help 
ensure consistent interpretation of the tasks and allow 
interviewers to probe for complete information. The 
final interview matrix form, while complex, detailed 


65 



task steps and the things often taken for granted, 
which are significant steps in teaching a course. It 
should be noted that although many faculty and staff 
acknowledged the distinct tasks in the matrix, they 
often could only recollect blocks of time spent on 
groupings of tasks. 


(Creating the survey) Involving faculty 

In the developmental stages of the survey, the team 
worked closely with a faculty associate in Distance 
Learning Services and several faculty familiar with 
the department and the Flashlight Program. These 
faculty were involved in creating the matrix form, 
drafting the request for participation, and running 
through mock interviews with the working model. 
Faculty participation gave the team an important 
perspective on how the study and the survey would 
come across to the study group. It was critical in 
helping define the language used in the survey to 
ensure it was the language used by faculty to describe 
their course work and tasks. It was also key to getting 
“buy in” for this project. 

Presenting the survey to faculty 

The matrix evolved from a simple, one page essay- 
style presentation to the more complex and detailed 
interview matrix with cover letter produced by the 
team. The cover letter was an invitation to 
participate. It introduced the survey to faculty and 
noted their experience with the Distance Learning 
process. It spoke of their support of Distance 
Learning Services and the recent adoption of the 
growth plan outlined in the Distance Learning Task 
Force Initiative. It described the importance of 
faculty participation in our ongoing study to improve 
processes and procedures. 


Conducting mock interviews with faculty 

With the matrix in a final draft stage and the letter 
developed, a staff member and faculty member were 
asked to participate in mock interviews. This gave the 
team a chance to become more familiar with the 
material, hear how participants may respond and 
develop responses to what may be difficult or 
confusing topics or issues. It was at this point that the 
importance of defining terms became an issue the 
team had to solve. 


Developing standard terms 

It was important to define terms and time definitions 
to ensure consistency in the interviews. Documenting 
information and placing it in the correct sections of 
the matrix was also essential for consistent data 
collection. The team defined the six major activity 
categories and three time definitions as follows: 

Activity categories 

Preparation - the time needed to prepare for course 
delivery. It involved the assessment of a 
faculty’s expertise using the format equipment 
and software; understanding Distance Learning 
processes and procedures; and general faculty 
preparation for delivery of an existing course. 

Presentation - the preparation needed for the course 
materials. It included all elements of planning 
lectures, gathering materials, scheduling short- 
term production sessions (for updates to the 
original materials produced up-front) and 
distributing materials. 

Interaction - the class time, after class discussions, 
chat sessions, office hours and travel to sites if 
appropriate. 

Assessment - tests and quizzes. 

Practice/application - special projects, papers and 
labs. 

Evaluation - the process of developing, distributing 
and assessing course evaluations. 

Time definitions 

Per quarter -A one-time allotment of time spent per 
quarter on a specific activity or task. 

Per week - This seemed to be the easiest breakdown 
for tasks because faculty and staff could recall 
the time per day or week they spent on a certain 
task. 

Per student - This was asked in reference to 
evaluation of student work such as tests, quizzes 
and projects/papers. Faculty members were 
asked to determine time spent evaluating a quiz, 
for example, and time per student. This provided 
at least two ways to evaluate this time — it would 
be obvious that more time would be spent on a 
larger class of students. 
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Conducting faculty interviews 

During interview sessions, the background of this 
project was discussed briefly. The faculty and staff 
interviewed were told more about the need for the 
information, how it would be used and a general time 
line for gathering and processing the information. 
One of the more important elements was 
confidentiality. Each faculty member was assured 
that information would be discussed with only the 
core group working on the matrix; all general 
information would be presented anonymously. No 
faculty would be referred to directly. Although the 
material would be made public, it was agreed that 
any notations would be general, such as “Information 
Technology course,” rather than “John Doe’s 
Telecommunications course.” 

Each member of our team was familiar with the 
participating faculty members and knew the elements 
of their courses. This background enabled us to 
uncover detailed information about the tasks and 
activities. There are many facets to delivering a 
course, yet to a faculty member, they are so common 
place that they are often taken for granted. 

It was important also to have a personal touch to this 
survey. A face-to-face discussion about 
confidentiality seemed to increase cooperation and 
good will. In hindsight, this method also served as the 
best way to gather significant anecdotal information. 
This material was an added dimension of the survey 
and reinforced, for the first time, some of the 
impressions only heard in informal conversations 
with faculty and staff. 

Tandem interviews were done for the first few 
sessions until the interviewing and reporting 
processes of the team became repeatable and routine. 
One member of the interview team would ask the 
questions while another wrote the answers. It was a 
way to have full concentration on what was being 
said and then to have a dual perspective on the 
information. It was also a back up situation in the 
case of intense, lengthy sessions. Several sessions 
went a half-hour; others took as long as two full 
hours. 


Conducting staff interviews 

Interviews were also conducted with staff in Distance 
Learning Services and in the academic departments. 
The team found that participants had difficulty 
separating time for individual faculty or individual 
course support. For example, when preparing for a 
new quarter, staff members contacted faculty through 
interoffice mail or e-mail as a group. Letters were all 
developed at once. Averages and percentages were 
used to tease out reasonable time estimates for 
individual courses. 

Cost Analysis Findings 

The sample provided us with information on a third 
of the KEY courses, one-sixth of the RAITN courses, 
and one-tenth of the other site-based courses. This 
gave us a good picture of the cost drivers of these 
site-based formats. However, the diversity in cost 
drivers in the anytime/anywhere format was not well 
represented by the three courses across three 
departments. Costs for courses in the on-campus 
format were unavailable at the detail required for the 
activity-based model. To get an approximation for 
comparison, we used the average cost per credit hour 
by college. Results indicated eight of the nine course 
sections in our sample cost roughly the same or less 
to deliver per credit hour as their on-campus 
counterparts (Table 3-2). 
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Table 3-2: Cost per credit hour 

(To ensure confidentiality, dollar amounts are adjusted to a fictitious base value in order to show relative 
relationships while concealing confidential financial data.) 


Course 

Average College Cost 
Per Credit Hour 

Site-Based Course Cost 
Per Credit Hour 

Anytime/ Anywhere 
Course Cost Per Credit 
Hour 

Engineering 

Technology 

$69 

$64 (E/MET) 


Engineering 

Technology 

69 

54 (E/MET) 


Environmental 

Management 

69 


$69 

Information 

Technology 

69 


30 

Mathematics* 

48 

48 

26 (E/MET) 
16 (KEY) 


Mathematics 

48 

109 (RAITN) 


Economics 

32 

10 (KEY) 


Political Science 

32 


35 


* Two distance learning sections of this course were included in the study. 


Comparing anytime/anywhere and campus formats 

Three courses in the sample group were offered both 
on-campus and in the anytime/anywhere format a 
political science course, an environmental 
management course and an information technology 
course. The three courses were from two different 
colleges. We compared the college’s average cost per 
credit hour for the campus format, with the cost per 
credit hour calculated by the model for each of the 
three anytime/anywhere courses (Table 3-2). 

Results showed that the political science and the 
environmental management courses had roughly the 
same cost per credit hour as their college averages 
(Table 3-2). In contrast, the information technology 
course was less than half (43%) of its college’s 
average cost per credit hour. This course also 
generated significant net revenue from graduate 
tuition. 

Other differences between the on-campus and 
anytime/anywhere formats of the same courses 
included differences in reported time by faculty in 
various course activities. For example, in the 
information technology course the faculty hours per 
student were roughly the same between the two 
formats (Table 3-5). The faculty member reported 
twice as many hours in preparation activities for the 
distance learning format. This increase, however, was 
balanced by fewer hours in presentation and 
interaction activities. In the environmental 
management course, the faculty member reported 


more than a 20% increase in hours per student in the 
distance learning version (Table 3-5). In this case, 
there were fewer hours reported for assessment 
activities but significantly more hours reported for 
interaction activities in the distance learning version. 

The political science course is an example of a course 
going through an incremental update. The faculty 
member reported more than two and a half times as 
much time per student in the distance learning 
version than the on-campus version of the course 
(Table 3-5). The increased hours reflect updating 
activities such as learning a new software product for 
class discussions; more time in identifying and 
gathering existing materials to incorporate into the 
course; creating new presentation materials; and 
additional hours evaluating student’s online 
participation. Reported hours in interaction, 
assessment, practice and evaluation activities were 
roughly the same in both formats. 

The three anytime/anywhere courses in the sample 
represent less than 3% of the courses in this format. 
The diversity of cost drivers is not well represented 
by such a small sample. The anytime/anywhere 
courses in the sample cost roughly the same or less 
than the average cost per credit hour for their colleges 
(Table 3-2). They also took faculty the same or more 
hours per student to deliver (Table 3-5). More study 
is needed using a larger sample to draw more 
comprehensive conclusions about the 
anytime/anywhere format. 


68 



Comparing site-based and campus formats 

Five courses (a total of six sections) in the sample 
were offered both on-campus and in a site-based 
distance learning format: two engineering technology 
courses, two mathematics courses, and one 
economics course. We compared the average 
college-specific cost per credit hour for the campus 
format with the cost per credit hour calculated by the 
model for each course. Results indicated that five of 
the six sections cost the same or less per credit hour 
in the distance learning site-based format (Table 3-2). 

The two courses with the lowest costs per credit hour 
were a mathematics course and an economics course 
offered to high schools using the Op tel Telewriter 
system (KEY program) (Table 3-2). One reason the 
costs to RIT were low is that the high school teachers 
oversaw most of the assessment and practice 
components of the courses. Technology costs were 
also low because the Telewriter data conferencing 
technology is older, low-cost, and used for many 
courses. On the revenue side, these courses broke 
even due to contract rates. 

One section of a site-based math course used 
two-way audio and video over fiber (RAITN 
program) and had the highest total costs per credit 
hour due to high capital and telecommunications 
costs (Table 3-2). The course’s total costs per credit 
hour were more than twice the costs per credit hour 
of its college average. The direct and indirect costs of 
the technology more than doubled the cost of the 
course. Contract rates were not sufficient to cover the 
costs. 


Two site-based engineering technology courses 
(E/MET program) used the Optel Telewriter 
technology. Their average cost per credit hour was 
15% less than the average cost per credit hour of 
on-campus courses in their college (Table 3-2). One 
section of a math course was also offered with this 
technology. It cost 46% less per credit hour than its 
college average (Table 3-2). These distance-learning 
courses had moderate total costs resulting from the 
costs for site coordinators and proctors and reported 
staff time to support site logistics. Two courses 
generated moderate net revenue from tuition. The 
other course generated high net revenue from a 
corporate contract that covered a significant portion 
of the costs. 

The distance-learning versions of the two engineering 
technology courses averaged 27% fewer faculty 
hours per student than their on-campus counterparts 
as seen in Table 3-5. Most of this time was savings in 
presentation activities since this component used 
videotapes. Faculty also reported spending relatively 
fewer hours in interaction activities in the distance 
learning versions. The math course consumed 25% 
more hours per student of faculty time, despite 
significant reductions in presentation, primarily due 
to increased preparation and travel to sites 
(Table 3-5). 

Comparing site-based and anytime/anywhere 
formats 

None of the courses in the sample were taught in both 
a site-based and anytime/anywhere format. However, 
we can make some general comparisons. The average 
cost of the anytime/anywhere courses and the 
site-based courses was roughly the same (Table 3-3). 
The site-based courses had higher average costs 
supporting interaction and assessment activities. The 
anytime/anywhere courses had higher average costs 
supporting preparation and practice activities. 
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Table 3-3: Average cost by activity 

(To ensure confidentiality, figures are adjusted to a base value that shows relative relationships.) 



Asynchronous Average Cost 

Site-Based Average Cost 

Preparation 

214 

68 

Presentation 

706 

665 

Interaction 

621 

847 

Assessment 

158 

324 

Practice 

266 

103 

Evaluation 

10 

3 

Total Costs 

$1,975 

$2,010 


Looking at cost type also illustrates the difference between the two formats (Table 4). Costs for the 
anytime/anywhere format are primarily faculty time. Costs in the site-based format are primarily other costs such as 
site fees and technology costs. The biggest differences occur because the site-based courses had average “other” 
costs nearly two and half times those of the anytime/anywhere format. Courses in the anytime/anywhere format, on 
the other hand, had 66% higher average faculty costs than the site-based courses. Some of this increase in faculty 
time is due to the format and some of it reflects the different nature of the courses themselves. 

Faculty time 

Faculty time was the largest and most variable portion of the total cost. Faculty’s reported time per student varied by 
more than a factor of three across the on-campus courses and by a factor of seven across the distance-learning 
courses (Table 3-5). In addition, the campus versions of the anytime/anywhere courses average 44% more reported 
faculty time per student than the campus versions of the site-based courses (Table 3-5). 

One surprising finding was faculty’s perception of time. Comparing different formats of the same course, the entire 
faculty felt they spent more time teaching the distance learning versions of their courses. The totals of the self- 
reported hours, however, indicated that only three of the nine distance learning sections consumed significantly 
more time per student (6-16 hours more per student). 

Five sections consumed roughly the same as the on-campus versions (1-3 hours difference). One section consumed 
significantly fewer hours per student (10 hours less) in the distance-learning version (Table 3-5). 


Table 3-4: Average cost by type 

(To ensure confidentiality, figures are adjusted to a base value that shows relative relationships.) 



Anytime/Anywhere 
Course Costs 

% 

Site-Based 
Course Costs 

% 

Other Costs 

450 

23% 

1107 

55% 

Faculty costs 

1362 

69% 

822 

41% 

Staff Costs 

163 

8% 

81 

4% 

Total Costs 

$1,975 

100% 

$2,010 

100% 
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Table 3-5: Reported faculty hours per student 

(To ensure confidentiality, figures are adjusted to a base value that shows relative relationships.) 



On-Campus Course 
Hours Per Student 

Site-Based Course 
Hours Per Student 

Anytime/Anywhere 
Course Hours Per 
Student 

Engineering Technology 

27 hrs. 

17 hrs. (E/MET) 


Engineering Technology 

17 

15 (E/MET) 


Environmental Management 

27 


33 hrs. 

Information Technology 

18 


17 

Mathematics* 

8 

8 

10 (E/MET) 
5 (KEY) 


Mathematics 

8 

36 (RAITN) 


Economics 

7 

5 (KEY) 


Political Science 

8 


20 


In analyzing what faculty members were doing 
differently in their distance learning classes, the 
differing activities included developing weekly study 
guides for students as additions to the syllabus; for 
the more lab-based courses, adding more problem 
examples in written supplemental guides and more 
class demonstrations in videotaped lectures. One 
faculty member stated that she developed more tests, 
quizzes and writing assignments in distance learning 
settings. She believed she could better assess student 
understanding of material with the additional 
materials since she could not see students in a 
classroom where a confused look is more evident. 

Some of the faculty also prepared additional 
materials for their distance learning classes, which 
enhanced the on-campus sessions as well. Others 
noted that that some of their time was spent 
presenting not only content material but also use of 
technology. Some of this additional course 
presentation was on electronic research techniques, 
along with training newcomers use equipment. 

The wide variety in how faculty reported spending 
their time in the different course delivery activities 
raises more questions for further study. Larger 
samples in focused curriculum will enable inquiry 
into the nature of faculty and student interactions, 
perceptions of time, and potential efficiencies of the 
different formats. 


Conclusion 

The primary purpose of this preliminary study was to 
gain a better understanding of our operating costs. 
The findings have been very useful in identifying the 
major differences in activities and costs between the 
different formats of distance learning. However, 
further study is needed to draw conclusive 
comparisons between distance learning and on- 
campus formats. More work is required to obtain cost 
data at the activity level for on-campus courses. A 
larger sample is also required to better understand the 
anytime/anywhere format. In addition, future studies 
may want to consider another method of time study 
instead of using self-reported time for faculty and 
staff. 

Results from this preliminary cost analysis have 
provided the following answers to the key business 
questions posed at the start of this study. 

Identify all costs associated with teaching a variety 
of distance learning courses. 

Through this study we have identified the various 
cost types and cost drivers in distance learning. We 
have gained a stronger understanding of our direct 
and indirect costs and we have a preliminary 
perspective on staff and faculty time. 
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Compare the costs of anytime/anywhere and 
site-based formats. 

The anytime/anywhere format appears to have cost 
advantages over other formats primarily because it 
doesn’t require sites and their associated staff and 
equipment costs. Further study with an expanded 
scope and a larger sample is needed for a better 
understanding of key cost drivers in this format 
including faculty time, costs to support interaction, 
and up-front development costs. 

Determine costs associated with distance learning 
staff time and services in order to develop a “price 
sheet" for departmental charge backs. 

Through this study we have identified the range of 
staff services we provide and have a glimpse into the 
time they consume. More comprehensive and 
accurate staff time reports are needed in order to 
develop a complete price sheet. 

Determine costs associated with teaching 
on-campus versions of the selected courses. 

Data for on-campus instruction beyond the Institute- 
wide averages were not readily available. We used 
average cost per credit hour by college as our cost 
comparison. We were not able to compare apples 
with apples. 

Identify revenue benefits of distance learning 
courses. 

Revenue and cost data suggest that graduate courses 
in the anytime/anywhere format offer the largest net 
revenue, followed by corporate contracts that exceed 
the high costs of site-based programs. 


Faculty Observations about the 
Anytime/ Anywhere Format 

This preliminary survey was essential for gathering 
data about costs of delivery. It also served as a way to 
learn about other factors that influence a department 
supporting or not supporting distance courses. Here 
are some faculty comments that illustrate the 
advantages and disadvantages of the 
anytime/anywhere format. 

Two faculty recalled on-campus students who had 
previously taken distance learning courses. They 
found that these students spoke up more in Distance 
Learning than they ever did in the classroom setting. 


The survey also brought out discussion about 
expectations of timely responses and concerns about 
computer literacy. “Students think that e-mail is 
instantaneous and that faculty respond as soon as 
they get the messages from students, ” said one 
faculty member. Student expectations of response 
times are higher in the distance formats than on- 
campus. 

“This is communication of the future and people need 
to see it that way, ” said one faculty member. The 
same principles of communicating, presenting, and 
articulating are as necessary to the distance format as 
on-campus. This faculty perceived some of his 
students as taking advantage of the freedom of 
Distance Learning, but without the discipline needed 
to succeed. 

In reference to distance learning courses as more time 
consuming to faculty, some faculty found that 
remedial work was required in other basic studies. 
One student told his professor when corrected 
because of his poor use of grammar and spelling 
errors, “ this is not a grammar class. ” Three faculty 
members emphasized that they are getting sub-par 
work, even in the graduate classes, and must advise 
students about writing basics. Perhaps some of the 
time that faculty believe they are spending in the 
distance format is remedial work, or perhaps the 
format itself is such that it emphasizes writing skills 
or the students’ lack of them. 

Some faculty members found that the FirstClass 
conference was a good way to see the progression of 
student knowledge. Conference submissions by 
students were a good way to see not only how much 
they participate, but also “ the richness of their 
conversations ” (meaning the depth of their 
understanding of the materials), said one faculty 
member. 

“Flexibility” and “control” were terms used 
repeatedly by faculty to describe success in the 
Distance Learning format. “Faculty must give up 
control as an autocrat in front of class, students 
define structure for themselves, ” said one faculty 
member. 

Finally, the survey brought out concerns about 
faculty promotion and compensation. Some faculty 
discussed the concepts of good teaching vs. faculty 
development: what constitutes development - 

increased content knowledge through research or 
development of new and improved teaching methods 
of delivery? Does the increase in technological 
expertise advance the faculty on the tenure track, or 
is it seen as less than valuable; are research of content 
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material and publishing the most important factors in 
promotion? 

Recommendations for the Model 

The IUPUI model is a strong and flexible foundation 
for developing individual school models. The ability 
for us to interact with the IUPUI model designers was 
important to the development of our individual 
application of the model. We recommend that there 
be some involvement with the IUPUI staff as 
consultants to develop common matrices, to define 
essential terms and concepts and to act as advisors for 
the costing data. 

The basics of the model and the how-to sections are 
clear. They are simple enough to get the concept. For 
our purposes, we could have used three things: 


Task development: we developed very detailed 
tasks following our instructional design model. 
However, faculty and staff reported back in numbers 
that grouped the tasks together. Having a common 
understanding of terms was critical for gathering 
accurate data and we spent a lot of testing out the 
terms we used. Suggestion: more detailed examples 
and suggested vocabulary. 

Data collection: we chose to interview rather than 
use questionnaires because of the complexity of our 
tasks and to make sure the terms were clear. Through 
this process we also gathered valuable anecdotal 
information. Suggestion: example survey questions. 

Time studies: we relied on self-reported time 
estimates for time on task that can lead to unreliable 
conclusions. Suggestion: information on more 

scientific time-study methods. 
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CASE STUDY 4: 

George Mason University 

Comparing Costs of Distance and Campus-Based Courses: Applying NCHEMS and 

NACUBO Techniques 

John Milam, Ph.D. 

HigherEd.org 


Abstract: This study compares the net costs of online versus traditional delivery of pairs of 
courses in four disciplines at George Mason University in 1998-2000. The study diverges from 
the basic activity-based costing approach described in this Handbook in several respects. There 
is less interest in documenting the cost of specific tasks or activities, because data on faculty 
workload were not gathered at this level of specificity. Rather, there is a more direct 
application of the basic micro-costing techniques used in indirect cost recovery and supported 
in the literature of the National Center for Higher Education Management Systems (NCHEMS) 
and the National Association of College and University Business Officers (NACUBO). 


Relevant Literature 

Modeling the costs of educational uses of technology 
is well served by understanding the general costing 
literature in higher education. The NCHEMS 
approach to cost of instruction models has been in 
place for almost thirty years. It is perhaps most useful 
today, when institutions have the tools to build online 
data marts and data warehouses that merge financial, 
student, facilities, and course datasets. Similarly, the 
literature of NACUBO for indirect cost recovery and 
micro- and macro-costing is invaluable. NACUBO 
literature includes Cost Accounting in Higher 
Education: Simplified Macro- and Micro-Costing 
Techniques (Jenny, 1996); A Cost Accounting 
Handbook for Colleges and Universities (Hyatt, 
1983); and Meisinger’s (1994) College and 
University Budgeting: An Introduction for Faculty 
and Academic Administrators. 

Among the most important contributions of the cost 
literature are the concepts of: (1) enrollment data and 
departmental consumption/contribution; (2) space 
utilization and allocation costs; (3) revenue stream 
based on tuition and fees minus waivers and tuition 
discounting; (4) faculty workload; and (5) 
administrative overhead at the department, 
college/school, and institution- wide levels. This does 
not mean that only complex models should be built. 
But any model development ought to be done with a 
clear set of assumptions. If assumptions about 
revenue, for example, are to be ignored (perhaps 
because student tuition estimates and financial aid 
data are not available or are too complex to be 


analyzed within the timeframe and staffing level), 
this should be stated as a limitation of the model. 

Methodology 

A modified version of the Flashlight cost analysis 
methodology serves as the basis for this study. 

For the purposes of the GMU Model, three new steps 
are inserted and one is modified. The step “identify 
the resources” is broken into two - for direct and 
indirect costs. The step for ‘calculate costs for these 
activities’ is also broken into two, with additional 
data about enrollment via the induced course load 
matrix. A new step is added to calculate revenue 
stream based on enrollment, tuition and fees, and 
financial aid data. “Document activities and tasks” is 
modified by not asking faculty to estimate how much 
time is spent on specific activities. 

The revised steps of a cost model for analyzing 
online courses now include the following: 

Define the resource issues 

1 . Choose outputs and performance measures 
Document activities and tasks 

2. Gather faculty and staff workload data 

3. Collect data on direct costs 

4. Calculate data on hidden, indirect, or shared 
administrative costs 

Gather data on enrollment 

5. Calculate results for each activity 
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6. Calculate revenue stream 

7. Summarize the results 

Steps for a Cost Analysis of Online Courses 

1. Define the resource issues 

This study compared the net costs of online versus 
traditional delivery of the same course and subject 
matter of four courses, one in each of four different 
disciplines, in 1998-2000. Table 4-1 documents the 
framework for comparison of online and traditional 
classes. 

Table 4-1: Comparison of Online vs. 
Traditional Course Sections 


2. Choose outputs and performance measures 

Two performance indicators were chosen to compare 
an online and traditional version of the same course: 
total net cost per course section and total net cost per 
course credit hour. 

The number of course credit hours was determined 
by multiplying the number of students enrolled at the 
semester census date by the number of credits 
awarded for completion of the course. In order to 
estimate net costs, data had to be collected about 
student course-taking patterns, residency status, and 
tuition and fees charges in order to calculate revenue 
stream. 


Semester 

Instructor 

Type 

Section 

Fall 99 

John 

Online 

ENGL300A 

Fall 99 

Jane 

Classroom 

ENGL300B 





Fall 99 

David 

Online 

MIS 200A 

Fall 99 

Sam 

Classroom 

MIS 200B 





Spring 99 

Mary 

Online 

DESC 200A 

Spring 99 

Louise 

Classroom 

DESC 200B 





Spring 99 

Iris 

Online 

ASTR100A 

Spring 99 

Peter 

Classroom 

ASTR100B 
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Table 4-2: Format for Analysis of Performance Measures 


Semester 

Type 

Section 

Net Cost 
per Section 

Net Cost per 
Course 
Credit Hour 

Fall 99 

Online 

ENGL300A 

$ 

$ 

Fall 99 

Classroom 

ENGL300B 

$ 

$ 






Fall 99 

Online 

MIS 2 00 A 

$ 

$ 

Fall 99 

Classroom 

MIS 200B 

$ 

$ 






Spring 99 

Online 

DESC 200A 

$ 

$ 

Spring 99 

Classroom 

DESC 200B 

$ 

$ 






Spring 99 

Online 

ASTR100A 

$ 

$ 

Spring 99 

Classroom 

ASTR100B 

$ 

$ 


3. Document activities and tasks 

The model developed in this paper did not involve 
gathering faculty estimates of the amount of time 
spent on different activities involved in teaching the 
various formats and subject matters. This feature was 
simply not practical without extensive faculty effort 
and record keeping. 

Because online courses are developed over time, an 
amortization schedule was used to divide 
development costs across multiple offerings of the 
course. Each course amortization schedule is unique, 
based on the number of times the particular class 
would be offered and the degree to which the 
developed technology was likely to continue to 
change and require additional input/costs. 


4. Gather faculty and staff workload data 

Table 4-3 documents estimates by each faculty 
member about the percent of their time, which goes 
for instruction-related activities and the percent of 
this instructional time which is devoted to the 
specific course. For several of the faculty who are 
part-time, this will be 100% in each column, as they 
have no other assigned duties under the teaching 
contract. They may teach other classes, but these are 
usually covered under separate contracts. 


Table 4-3: Faculty Workload for Instruction and Section 


Semester 

Instructor 

Type 

Section 

Percent for 
Instruction 

% of Instruction 
for Section 

Fall 99 

John 

Online 

ENGL300A 

50% 

30% 

Fall 99 

Jane 

Classroom 

ENGL300B 

100% 

100% 







Fall 99 

David 

Online 

MIS 200A 

50% 

50% 

Fall 99 

Same 

Classroom 

MIS 200B 

100% 

100% 







Spring 99 

Mary 

Online 

DESC 200A 

50% 

25% 

Spring 99 

Louise 

Classroom 

DESC 200B 

40% 

50% 







Spring 99 

Iris 

Online 

ASTR100A 

65% 

30% 

Spring 99 

Peter 

Classroom 

ASTR100B 

50% 

50% 
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5. Collect data on direct costs 
Personnel Expenditures 

Direct costs such as faculty compensation, including 
salary and benefits, are relatively easy to obtain from 
the institutional research, budget, and human 
resource office. However, it is important to verify 
these data with the faculty involved. This is 
especially true for part-time faculty who may teach 
several classes, each paid with a different funding 
record on different dates. Human resource data for 
these faculty will vary by date and funding record, 
depending on the extract date of the census file used 
for analysis. 

Note: The term “extract file” or “census file” refers 
to a specific dataset that is “cut,” “extracted,” or 
created from the institution’s administrative 
information systems. Human resource data would 
come from the human resources information system, 


student and course data from the student information 
system, and finance data from the financial 
information system. While there are many vendors 
for these systems, such as SCT Banner, PeopleSoft, 
Datatel, and Oracle, the fundamental natures of the 
systems are the same. When gathering data about 
people, students, courses, and money, institutional 
researchers must first obtain an extract of these data, 
usually by requesting the computing staff to run a 
special program to “cut” data on a certain date and 
put them in a certain format. These extract files are 
then used by institutional research, planning, budget, 
and assessment offices for many different kinds of 
studies. 

Table 4-4 lists faculty compensation and benefits data 
for each of the comparison classes or sections. 


Table 4-4: Direct Faculty Compensation & Benefits per Section 


Section 

Salary 

Benefits 

Rate 

Total 

Compensation 

Comp. Per 
Semester 

% for 
Instr. 

Comp, for 
Instruction 

% for 
Section 

Comp. Per 
Section 

ENGL300A 

$61,435.00 

23.50% 

$75,872.23 

$37,936.11 

50.0% 

$18,968.06 

30.0% 

$5,690.42 


$2,400.00 



$2,400.00 




$2,400.00 










MIS 200A 

$102,962.00 

23.5% 

$127,158.07 

$63,579.04 

50.0% 

$31,789.52 

50.0% 

$15,894.76 

MIS 200B 

$5,310.00 

7.65% 

$5,716.22 

$5,716.22 


$5,716.22 


$5,716.22 










mmm 

$84,084.00 

23.50% 

$103,843.74 

$51,921.87 

EH 




BiSMi 

$72,300.00 

23.50% 

$89,290.50 

$44,645.25 

40.0% 

$17,858.10 

50.0% 











ASTR 100 A 

$60,400.00 

23.50% 

$74,594.00 

$37,297.00 

65.0% 

$24,243.05 

30.0% 

$7,272.92 

ASTR 100B 

$73,200.00 

23.50% 

$90,402.00 

$45,201.00 

50.0% 

$22,600.50 

50.0% 

$11,300.25 


Note: It is important to include other, non-faculty 
personnel who may be peripherally involved in 
offering the course. Examples include computing 
support staff who set up a special web server for the 
course or student wage employees who develop 


applications such as Java simulation exercises or 
ColdFusion™ templates administering online 
quizzes. Data on these costs were gathered with 
estimates from the faculty interviews conducted at 
the university being studied. 
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Table 4-5: Other Direct Personnel Costs for Online Sections 


Section 

Personnel 

Time 

Rate of Pay 

Cost 

ENGL300A 

Course Instructor 

Additional funding for 
course development 

7K Amortized over 1 2 semester 
sections 

$583.34 

ENGL300A 

MIS faculty 

1 0 hours 

60K + Benefits = $74,100 /2080 
hours - $35.62 per hour/ Amortized 
as above 

$29.69 

ENGL300A 

Research assistant 

■■ESSSflESE^Bni 

$2,000 Amortized as above 

$166.67 

ENGL300A 

Course consultant 

Spring 1999 

$3,000 Amortized as above 

$250.00 

ENGL300A 

Course consultant 

Spring 1999 

$2,000 Amortized as above 

$166.67 

ENGL300A 

Assessment developer 

Spring 1999 

$2,000 Amortized as above 

$166.67 

ENGL300A 

Total 


Total 

$1,976.05 






MIS 200A 

Course Instructor 

Additional funding for 
course development 

$23,000 Amortized over 6 
semesters 

$3,833.33 

MIS 200A 

Eric 

50 hours in semester 

$ 1 3 per hour 

$650.00 

MIS 200A 

Eric 

20 hours per week in 
summer/ 10 weeks=200 

$ 1 3 per hour. Amortized over 6 
semesters 

$433.00 

MIS 200A 

Jeff 

1 0 hours 

$ 1 3 per hour. Amortized 

$22.00 

MIS 200A 

Jenny 

1 0 hours 

$ 1 3 per hour. Amortized 

$22.00 

MIS 200A 

Eric’s brother 

One-time 

$50 one-time 

$50.00 

MIS 200A 

Total 


Total 

$5,01033 






DESC 200A 

Course Instructor 

Additional funding for 
course development 

$8,000 Amortized over 6 semesters 

$1,333.33 

DESC 200A 

TA for Spring 97 

20 hours per week * 1 5 
weeks = 300 hours 

$ 1 0 per hour. Amortized 

$500.00 

DESC 200A 

TA in Summer for 
Java 

N/A 

$3,000. Amortized 

$500.00 

DESC 200A 

Total 


Total 

$2,33333 






ASTR 100 A 

Course Instructor 

Additional funding for 
course development 

$15,000 Amortized over 
6 semesters 

$2,500.00 

ASTR 100 A 

Total 


Total 

$2,500.00 


Non-Personnel, Direct Expenditures 

A breakout of non-personnel expenditures is documented in the following table. These are amortized over time 
where appropriate. For two of the online courses, there are no expenditures of this type. The traditional course 
counterparts also have no such expenditures. 
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Table 4-6: Direct Non-Personnel, Non-Computing Costs for Online Sections 


Section 

Expenditure Type 

Total Cost 

Amortization 

Section 

Cost 

ENGL300A 

Books 

$200 

Over 6 semesters 

$33.33 

ENGL300A 

Conference Travel 

$3,000 

Over 6 semesters 

$500.00 

ENGL300A 

Total 


Total 

$550.00 






MIS 200A 

Overheads 

$10 

N/A 

$10.00 

MIS 200A 

Cassette Tapes 

$1 per tape X 80; 20% 
lost per semester 

$80 over 6 semesters; 
$20 extra per semester 

$33.33 

MIS 200A 

Audio Duplication 

$1 per tape X 80; 20% 
lost per semester 

$80 over 6 
semesters; $20 extra 
per semester 

$33.33 

MIS 200A 

Total 


Total 

$76.66 






DESC 200A 

N/A 

N/A 

Total 

$0.00 






ASTR 100 A 

N/A 

N/A 

Total 

$0.00 


6. Calculate data on hidden, indirect, or shared 
administrative costs 
Departmental Overhead 

Data were obtained for each department and division 
in which course sections were offered: English 
(ENGL), Management Information Systems (MIS), 
Decision Sciences (DESC), and Astronomy (ASTR). 
For ENGL, there is a clear cost center in the 
department of English. Examining the account 
structure for MIS and DESC, the data show that only 
faculty salary and fringe benefits are pooled at the 
department level. It is therefore necessary to make 
calculations about these courses based on expenses 
and activity at the deans ’/division level. ASTR is an 
interesting case, because the faculty member teaching 
the online course is in another smaller division that 
does not have separate department account structures. 
Cost analysis of the ASTR course must include a 
traditional version of the course, which in this case is 
offered in the department of Physics/ Astronomy. This 
requires that departmental and division overhead 
charges be calculated for this comparison section. 

Administrative overhead costs at the department level 
need to be calculated only for English and 
Physics/Astronomy. Division level overhead costs 
need to be calculated for the School of Management 
(SOM) where MIS and DESC are housed, the 
Institute for Computational Science and Informatics 
(ICSI) where the online astronomy faculty is located, 
and the College of Arts and Sciences (CAS) where 
the ENGL and Physics and Astronomy departments 
are housed. In each case, it is necessary to obtain 


financial expenditure data at the object code level 
within all instructional accounts related to each cost 
center. Also, since the ASTR and DESC sections 
were offered in spring 1999, overhead must be 
calculated for the 1998-99 academic year. The MIS 
and ENGL sections require overhead data for 
1999-00. 

In examining departmental overhead, it is necessary 
to delete expenses related to the other full-time and 
part-time faculty and graduate assistants who have no 
responsibilities for the courses in this model. The 
model should only prorate costs that may reasonably 
be associated with overhead for a course section. 
Only the overhead functions of administrators and 
classified/non-faculty staff should be included. 
Because fringe benefits costs are lumped together in 
one series of object codes at this university, it is 
necessary to calculate fringe costs separately. Per the 
Budget Director, these are estimated as 23.5% fringe 
for administrators and non-faculty staff and 7.65% 
for wage employees. There is no fringe benefit 
calculation for part-time, temporary, college work 
study, and student wage employees. 

“Other than personnel” services (OTPS) expenditures 
are subtotaled separately. All types of instruction- 
related, non-personnel expenses such as copying and 
telephones that may not be directly tied to a specific 
course should be including in this estimate. If direct 
expenditures are included for the course and these 
were paid from the same account as the department 
or division account, then these should be subtracted 
from the total so as not to be counted twice. 
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After deleting faculty data and estimating fringe 
costs, the data on expenditures by object code appear 
in Table 4-7 for the English Department. 


Table 4-7: Partial Expenditures by Object 
Code for English Department 


Object Code 

Description 

Budget 

1310 

CLASSIFIED SALARY 

109,838 


Fringe for classified 

25,812 

1400 

BUDGET POOL-WAGES&OT 

68,989 

1420 

WAGES-STUDENTS 

6,111 

1610 

STUDENT WAGES CWS 

5,000 

2000 

BUDGET POOL-OTPS 

24,004 

3110 

EXPRESS SERVICES 

186 

3140 

METERED MAIL 

3,241 

3144 

BULK MAIL 

20 

3148 

OTHER MAIL 

271 

3150 

PRINTING SERVICES 

14 

3160 

TELECOMM SVS DIT 

29,100 

3170 

TELECOM SVS NONSTATE 

7,245 

3177 

LONG DISTANCE TELE 

2,063 


These expenditures by object code are aggregated in 
sub-groups and totaled for each department. Table 
4-8 shows the data for the Physics/ Astronomy and 
English departments. 


Table 4-8: Expenditures by Sub-Group by 
Department 


Departmental Budget 

1998-99 

PHYS/ASTR 

1999-00 

ENGL 

Classified Staff 

54,418 

105,200 

Wages 

8,296 

0 

Student Wages 

7,446 

6,111 

College Work Study 

7,500 

5,000 

Fringe - Classified/ Admin Fac 

12,788 

24,722 

Fringe - Wages 

635 

0 

OTPS i 

55,983 

121,136 

Total Expenditures 

147,066 

262,169 


As stated earlier, there are many ways to allocate 
overhead costs. A unique allocation scheme is 
necessary for the purposes of the online versus 
traditional course comparison. Usually, this 
calculation is done using student or course credit 
hours or major headcount. Credit hours are chosen 
for some models of departmental activity because 
headcount implies costs related to advising majors. 
Since decision-making, enrollment planning, content 
management, and staffing all take place at the level 


of the individual course section, this is the 
methodology chosen for this particular example 

To allocate or prorate departmental expenditures per 
course section, it is necessary to know how many 
sections were offered during the academic year. This 
must include both fall and spring semesters. Summer 
enrollment, faculty contracts, and twelve-month 
administrative staffing in departments and deans’ 
offices are unique issues not dealt with here, though 
they are important for analysis of online courses 
offered or developed during the summer months. 
Because all of the courses being studied were offered 
in either spring 1999 or fall 1999, the number of 
course sections is calculated accordingly. 

The institutional research office of the university 
collects data on course section offerings for mandated 
state reporting and internal analyses. An existing 
series of enrollment reports capture these data at the 
division level, listing the number of sections by type 
offered by each division. Using the GMU Data 
Warehouse, these data were obtained for the four 
semesters of fall 1999, spring 1999, fall 1999, and 
Spring 2000 in order to estimate costs per section for 
the 1998-99 and 1999-2000 academic years. Special 
runs of the same report were required for the data at 
the department level. The data on departmental 
sections for English appear in Table 4-9. 


Table 4-9: Total 1999-2000 Section Data for 
the English Department 


English 

Fall 

1999 

Spring 

2000 

Total 

Lecture/Seminar Sections 

265 

270 

535 

Lab/Rct Sections 

1 

1 

2 

Total 

266 

271 

537 


These data were then used in conjunction with the 
expenditure data to calculate estimated departmental 
overhead costs per course section. 

Table 4-10: Expenditures per Course Section 
for the English Department 


Department Budget 

ENGL 

1999-00 

Total Expenditures 

$262,169 

Total number of sections 

537 

Dept overhead per section 

$488.00 
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Deans’/Division-Level Overhead 

Only those accounts directly related to administrative 
operations should be included in this second set of 
calculations for dean/division-level overhead. These 
accounts were identified by using financial reports 
from the GMU Data Warehouse and verified by the 
GMU Budget Office. Again, only object codes for 
non-personnel (OTPS) expenditures, classified/non- 
faculty staff, and administrators are included. Other 
expenses for full-time and part-time faculty and 
graduate assistants were excluded because they do 
not contribute to the administrative overhead 
necessary to offer the courses. Data were combined 
by object code across accounts. Then fringe benefits 
were estimated using the same method described 
above for departments. 


Table 4-11: Administrative Overhead 
Accounts within a Division 


Account 

Name 

111274 

IT&E LAB SUPPORT 

111275 

IT&E PT/GTA 

111997 

RESERVE DEAN IT&E 

146010 

DEAN IT&E 

146011 

IT&E CONTRACT COURSE SHARE 

154075 

IT&E GRAD ADMISSIONS 


Table 4-12: Expenditures by Sub-Group by Division 


Expenditure 

ICSI 

SOM 

SOM 

ITE 

CAS 

CAS 

Division Budgets 

1998-99 

1998-99 

1999-00 

1998-99 

1998-99 

1999-00 

Administrative Faculty 

0 

436,310 

822,675 

571,567 

672,837 

812,900 

Classified Staff 

197,161 

670,434 

771,647 

187,171 

399,099 

410,386 

Wages 

0 

11,438 

17,144 

61,531 

25,022 

19,102 

Student Wages 

0 

41,552 

20,409 

69,269 

4,995 

8,092 

College Work Study 

7,500 

15,000 

12,000 

6,000 

3,600 

3,000 

Fringe - Classified/ Admin Fac 

46,333 

260,085 

374,666 

178,303 

251,905 

287,472 

Fringe - Wages 

0 

875 

1,312 

4,707 

1,914 

1,461 

OTPS 

124,798 

410,895 

836,714 

389,199 

249,630 

389,046 

Total Expenditures 

375,792 

1,846,589 

2,856,566 

1,467,748 

1,609,002 

1,931,460 
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Because services increase with increased student 
course load, the division and institutional overhead 
allocations are made by course credit hours, not by 
course sections. These data are obtained from the 


“Glimpse of Course Enrollment” reports that are 
produced each semester by the University’s Office of 
Institutional Research and Reporting. Division data 
for one semester appears in Table 4-13. 


Table 4-13: Fall 1999 Course Sections by Division 


Academic 

Division 

Course Level 

Lec/Sem 

Sections 

Lab/Rct 

Sections 

Credit 

Hours 

Total FTE 

CAS 

Undergrad 

1,039 

296 

118584 

7905.6 

CAS 

Graduate 

220 

15 

10559 

879.9 

CAS 

Total 

1,259 

311 

129143 

8785.5 

ICSI 

Graduate 

25 

0 

659 

54.9 

ICSI 

Total 

25 

0 

659 

54.9 

ITE 

Undergrad 

93 

88 

12975 

865 

ITE 

Graduate 

106 

0 

7737 

644.7 

ITE 

Total 

199 

88 

20712 

1509.8 

SOM 

Undergrad 

153 

2 

25055 

1670.3 

SOM 

Graduate 

41 

0 

2572.5 

214.4 

SOM 

Total 

194 

2 

27627.5 

1884.7 


The data must be aggregated across semesters to 
obtain total course credit hours generated in the 
academic year, broken out by division. With these 


results, division overhead costs per section and per 
course credit hour were then be calculated as in Table 
4-14. 


Table 4-14: Overhead Cost Calculations by Division 



1998-99 

ICSI 

1998-99 

SOM 

1999-00 

SOM 

1998-99 

ITE 

1998-99 

CAS 

1999-00 

CAS 

Total Expenditures 

375,792 

1,846,589 

2,856,566 

1,467,748 

1,609,002 

1,931,460 

Total Sections 

45 

409 

419 

570 

2,991 

3,124 

Division Overhead per 
Section 

8,351 

4,515 

6,818 

2,575 

538 

618 

Course Credit Hours 
(Academic Year) 

1,091 

54,938 

56,329 

40,108 

242,144 

248,685 

Division Overhead per 
Course Credit Hour 

$344.45 

$33.61 

$50.71 

$36.59 

$6.64 

$7.77 
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Table 4-15 uses the calculated cost per course credit 
hour by division and applies it to the four pairs of 
comparison sections. 


There is wide variation in overhead costs per course 
credit hour. The ICSI division, which produces far 
fewer sections, is much more expensive. Notice that 
deans’/division-level overhead is an important cost 
driver. 


Table 4-15: Division Overhead Costs per Online Section 


Overhead Calculations 

Online 

Classroom 

Online 

Classroom 

Online 

Classroom 

Online 

Classroom 

Class Section 

ENGL300A 

ENGL300B 

MIS200A 

MIS200B 

DESC200A 

DESC200B 

ASTR100A 

ASTR100B 

Headcount Enrollment 

13 

22 

76 

81 

85 

62 

48 

306 

Course Credit Hours 

39 

66 

228 

243 

255 

186 

144 

918 

ISHiBI 

7.77 

7.77 

50.71 

50.71 

33.61 

33.61 

344.45 

6.64 

Division overhead 
for online section 

302.90 

512.60 

11,562.38 

12,323.06 

8,571.12 

6,251.88 

49,600.39 

6,099.94 


Institutional Overhead 

To calculate overhead at the institution level, it was 
first necessary to separate academic divisions that 
generate courses from those purely administrative or 
operational units such as student affairs, computing, 
and libraries. Using the GMU Data Warehouse, 
financial data on budgets for expenditures were 


obtained for the two years during which course 
sections met, 1998-99 and 1999-00. The list of 
divisions and expenditures appears in Table 4-16. 
After the data on budget by division are totaled and 
the course credit hours offered over the two 
semesters of each academic year totaled, the cost per 
course credit hour was calculated. 
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Table 4-16: Institutional Overhead Budgets by Division 


Division 

1998-99 Budget 

1999-00 Budget 

Balance Appropriation 

3,982,301.00 

4,328,178.00 

Multi-Campus 

783,712.00 

773,026.00 

Budget & Planning 

1,008,812.00 

1,019,749.00 

Campus Life 

543,068.00 

681,530.00 

Instr. Improvement & Technology 

2,910,801.00 

3,277,183.00 

Enrollment Services 

3,640,611.00 

3,807,455.00 

Senior Vice President 

1,469,411.00 

1,188,887.00 

Fiscal Services 

3,239,573.00 

3,412,583.00 

Human Resources 

1,553,217.00 

1,658,936.00 

Instructional Support 

2,581,769.00 

2,479,951.00 

Information Technology & Support 

382,133.00 

1,597,316.00 

Library 

8,992,581.00 

9,612,538.00 

Operations 

2,648,634.00 

3,278,338.00 

Plant/Facilities 

12,878,921.00 

13,707,845.00 

Executive Affairs 

1,559,141.00 

1,708,639.00 

Academic Administration 

4,088,030.00 

4,636,672.00 

Property Rental 

1,095,000.00 

2,080,000.00 

Student Development 

2,048,111.00 

2,343,562.00 

Safety Operations 


566,382.00 

University Life 

1,239,911.00 

1,323,725.00 

University Development 

1,108,453.00 

1,221,669.00 

University Computing & Info Systems 

9,857,663.00 

11,197,084.00 

University Relations 

2,632,316.00 

2,642,767.00 


Using the total budget per year in institutional 
overhead made it possible to calculate the cost per 
course credit hour. For this performance measure, the 


number of course credit hours in all divisions was 
totaled for both semesters in each academic year. The 
results appear in Table 4-17. 


Table 4-17: Institutional Overhead Calculations per Course Credit Hour 


Division 

1998-99 Budget 

1999-00 Budget 

Total expenditures 

$70,300,697.00 

$78,601,221.00 

Academic Year Course Credit Hours 

444,779.00 

451,152.00 

Cost per course credit hour 

$158.06 

$174.22 


After the cost per course credit hour was calculated, overhead costs for each section as seen in Table 4-18. 

those figures were used to obtain the institutional 
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Table 4-18: Institutional Overhead Costs per Section 


Overhead Calculations 

Online 

Classroom 

Online 

Classroom 

Online 

Classroom 

Online 

Classroom 

Class Section 

ENGL300A 

ENGL300B 

MIS200A 

MIS200B 

DESC200A 

DESC200B 

ASTR100A 

ASTR100B 

Headcount Enrollment 

13 

22 

76 

81 

85 

62 

48 

306 

Course Credit Hours 

39 

66 

228 

243 

255 

186 

144 

918 

Institutional overhead per 
course credit hr 

174.22 

174.22 

174.22 

174.22 

158.06 

158.06 

158.06 

158.06 

Institutional overhead 

6,794.71 

11,498.74 

39,722.93 

42,336.28 

40,304.69 

29,398.71 

22,760.29 

145,096.87 


Space Costs 

The first step for this part of the model was to obtain 
an “extract file” from the University that contained 
course meeting data for the semester in which the 
section met. Using data on the stop and start dates for 
which the class was offered, the days of the week the 
class met, and the beginning and end meeting times, 
it was possible to calculate the amount of time each 


class met per day, week, and semester. This total time 
per class per semester was then calculated as a 
percentage of the total time in which a room was 
utilized for instruction to prorate space costs. A 
portion of a course meeting file for the university is 
displayed in Table 4-19. 


Table 4-19: Course Meeting File for Classroom Utilization Statistics 


Term 

Disc 

CNUM 

Sect 

Start 

Stop 

Days 

BLDG 

Room 

Begin 

End 

Class 

Time 

Weekly 

Time 

Total 

Time 

99F 


122 

1 

8/30/39 

9/29/39 

TR 

ENT 

275 

9:00 

10:15 

1:15 

2:30 

10:42 

99F 


123 

1 

10/4/39 

11/3/39 

TR 

ENT 

275 

9:00 

10:15 

1:15 

2:30 

mu 

99F 


602 

1 

9/9/39 

10/14/39 

F 

ENT 

275 

18:00 

21:30 

3:30 

3:30 

mm 

99F 

LRNG 

602 

1 

9/17/39 

10/22/39 

S 

ENT 

275 

9:00 

16:30 

7:30 

7:30 

37:30 

99F 

PUAD 

660 

2 

8/26/39 

10/7/39 

F 

ARL 

257 

18:00 

21:00 

3:00 

3:00 

18:00 

99F 

PUAD 

661 

1 

10/21/39 

12/9/39 

F 

ARL 

257 

18:00 

21:00 

3:00 

3:00 

21:00 

99F 

PUAD 

660 

2 

8/20/39 

10/8/39 

S 

ARL 

257 

9:15 

17:15 

8:00 

8:00 

56:00 

99F 

PUAD 

661 

1 

10/15/39 

12/10/39 

S 

ARL 

257 

9:15 

17:15 

8:00 

8:00 

64:00 


For the purposes of this study, the local commercial, 
space rental rate of $25 per square foot per year was 


used, prorated over the course of three semesters to 
$8.33 per square foot per semester. 











































































































































































Table 4-20: Space Costs per Course Section 


Section 

Room 

Sq Feet 

Cost per SF 
for semester 

Total cost 

Hrs used 

Total use 

%Use 

Space cost 

ENGL300A 

Online 

0 

8.33 

$0.00 

0:00 

0:00 

0.0% 

$0.00 

ENGL300B 

Robinson B222 

484 

8.33 

$4,031.72 

40:00 

981:20 

4.1% 

$164.34 



BMH 







MIS 200A 

Online/FAB B 1 08 


8.33 

$8,746.50 

6:00 

502:40 

1.2% 

$104.40 

MIS 200B 

AQ 102 


8.33 

$11,595.36 

40:00 

613:20 

6.5% 

$756.22 










DESC 200A 

Online/ AQ 102 


8.33 

$11,595.36 


632:25 

1.9% 

$220.02 

DESC 200B 

ST 131 


8.33 

$14,944.02 


732:51 

5.9% 

$885.34 










ASTR 100 A 

Online 

0 

8.33 

$0.00 

6:00 

0:00 

0.0% 

$0.00 

ASTR 100B 

LH 1 

2,556 

8.33 

$21,291.48 

40:42 

374:34 

10.9% 

$2,313.51 


Note that two of the online courses, ENGL300A and 
ASTR100A, did not meet at all on campus. The other 
two online courses met two and four times 
respectively on campus, usually at the beginning and 
end of the semester, for orientation and for exams. 

In addition to these classroom costs, it was important 
to estimate the amount of other space used by the 
faculty and course. In conducting interviews or 


surveys with faculty about the course, information 
was collected about other space they used. Where it 
was difficult to obtain these data, the same estimates 
used for percent of instruction devoted to instruction 
and to the individual course section were also applied 
to percent of use for faculty office space. Table 4-21 
illustrates the calculation of this non- 
classroom space. 


Table 4-21: Non-Classroom Space Costs per Course Section 


Section 

Room 

Type 

% Use 

Space Amt 

Calculation 

Cost 

ENGL300A 

Rob A421 

Office 

5.0% 

107 SF 

5% X ( 1 2 1 SF X 
$8.33 per SF) 

$50.40 

ENGL300B 

Part-time 

None 

n/a 

n/a 

n/a 

n/a 








MIS 200A 

ENT 146 

Office 

30.0% 

121 SF 

30% X (121 SF X 
$8.33 per SF) 

$302.38 

MIS 200A 

ICASIT 

Center 

Office 

n/a 

1 of 3 rooms 

1 room in Center (1/3 
X 1200) = $400 per 
moX 10% 

$40.00 

MIS 200A 

Total 




Total 

$342.38 

MIS200B 

Part-time 

None 

n/a 

n/a 

n/a 

n/a 








DESC200A 

ENT 152 

Office 

20.0% 

128 SF 

20% X (121 SF X 
$8.33 per SF) 

$213.25 

DESC200B 

ENT 144 

Office 

20.0% 

140 SF 

20% X (121 SF X 
$8.33 per SF) 

$233.24 








ASTR 100 A 

ST1 109 

Office 

20.0% 

118 SF 

20% X (121 SF X 
$8.33 per SF) 

$196.59 

ASTR100B 

ST1 213 

Office 

20.0% 

141 SF 

20% X (121 SF X 
$8.33 per SF) 

$234.91 
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Computing Support Costs 

Direct computing costs included in the model were 
the computer, printers, and network used by the 
faculty, staff, graduate assistant, and (sometimes) the 
students in the course. Some estimates were made 
from data about faculty workload, paralleling the 
calculation of faculty office space usage. For 
example, if 65% of a faculty member’s activities was 
instruction-related, then 65% of the personal 
computer cost was assumed to be instruction related. 
Suppose this faculty member taught four courses per 
academic year. It seemed reasonable to assume that 


16.25% (1/4 of 65%) of the amortized semester’s cost 
for purchasing the PC and 16.25% of direct software 
and other computing costs could be allocated to each 
of the four course sections. 

Another method for estimating direct computing 
costs is to interview each person involved in the 
course about the equipment and software and the 
percent of time used for the specific section. This 
second method was adopted for the comparison of 
online courses in this paper as seen in Table 4-22. 


Table 4-22: Direct Computing Costs per Section 


Section 

Type 

Total Cost 

Amortization 

Cost ner Section 

ENGL300A 

Cable modem 

$50 per mo 

None. 50% for class 

100.00 

ENGL300A 

Telephone line 

$20 per mo 

None. 50% for class 

40.00 

ENGL300A 

Computer 

2,500.00 

Six semesters, 25% for class 

104.17 

ENGL300A 

Scanning software 

100.00 

Six semesters, 25% for class 

4.17 

ENGL300A 

Total 


Total 

248.33 

ENGL300B 

None 

0.00 

None 

0.00 






MIS 200A 

Internet Service 

$50 per mo 

None. 5% for class 

10.00 

IS 200A 

Compaq Computer 

1,500.00 

Six semesters, 25% for class 

62.50 

MIS 200A 

Home Computer 

2,400.00 

Six semesters, 25% for class 

100.00 

MIS 200A 

Total 


Total 

172.50 

MIS200B 

None 

0.00 

None 

0.00 






DESC200A 

Computer 

2,000.00 

Six semesters, 50% for class 

83.33 

DESC200B 

Computer 

2,000.00 

Six semesters, 50% for class 

83.33 






ASTR100A 

Computer 

2,000.00 

Six semesters, 50% for class 

83.33 

ASTR100B 

Computer 

2,000.00 

Six semesters, 50% for class 

83.33 


Data on indirect computing costs such as Internet 
access and threaded discussion groups are also 
needed for the course sections. These data are very 
hard to collect and the allocation schemes necessary 
for prorating usage across courses or transactions are 


rudimentary at best. For the purposes of the model 
described in this paper, these indirect computing 
costs will be assumed to be included in the 
institutional overhead calculation for computing 
which is allocated per course credit hour. 
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7. Gather data on enrollment 


Table 4-23 of information about headcount and course credit hour enrollment was needed in order to calculate the 
two performance measures. 

Table 4-23: Enrollment per Section 


Section 

Headcount 

Course Credit 
Hours 

ENGL300A 

13 

39 

ENGL300B 

22 

66 ; 




MIS200A 

76 

228 

MIS200B 

81 

243 




DESC200A 

85 

255 

DESC200B 

62 

186 




ASTR100A 

48 

144 

ASTR100B 

306 

918 


8. Calculate results for each activity 

Table 4-24 lists each type of cost calculated in the model and provides the total expenditures per course section. 

Table 4-24: Total Expenditures per Section 


Expenditure 

ENGL300A 

ENGL300B 

MIS200A 

MIS200B 

C 



1 

ASTR100B 

Direct faculty salaries & 
benefits 

5,690.42 

2,400.00 

15,894.76 

5,716.22 

6,490.23 

8,929.05 

7,272.92 

11,300.25 

Other personnel costs 

1,976.05 

0.00 

5,010.33 

0.00 

2,333.33 

0.00 

2,500.00 

0.00 

Direct, non-personnel 
costs 

450.00 

0.00 

76.66 

0.00 

0.00 

0.00 

0.00 

0.00 

Departmental overhead 


488.00 

n/a 

n/a 

n/a 

n/a 

n/a 

1,149.00 

Division overhead 


512.60 

11,562.38 

12,323.06 

8,571.12 

6,251.88 

49,600.39 

6,099.94 

Institutional overhead 

6,794.71 

1 1,498.74 

39,722.93 

42,336.28 

40,304.69 

29,398.71 

22,760.29 

145,096.87 

Classroom space costs 

0.00 

164.34 

104.40 

756.22 

220.02 

885.34 

0.00 

2,313.51 

Office space costs 

50.40 

0.00 


0.00 

213.25 

233.24 

196.59 

234.91 

Direct computing costs 

248.33 

0.00 


0.00 

83.33 

83.33 

83.33 

83.33 










Total Expenditures 

16,000.81 

15,063.68 

72,886.33 

61,131.78 

58,215.96 

45,781.55 

82,413.52 

166,277.81 


9. Calculate revenue stream Tuition Revenue- 
based on Enrollment 

Table 4-25 shows a portion of an extract file is 
displayed to illustrate the calculation of tuition 


revenue based on credit hour load and residency 
status. This requires knowledge of the institution’s 
tuition policy, particularly for out-of-state students 
who take the maximum number of hours permitted 
without paying additional tuition charges per credit. 
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Table 4-25: Section Enrollment by Residency, Hours Taken for Revenue Stream 


TERM 

Section 

Level 

Residency 

Total 

Hrs 

Taken 

Total 

Tuition 

Charges 

Percent of 
Hours for 
Section 

Class 

Tuition 

99B 

ASTR100 

FR 

In-State 

3 

$454.50 

100.00% 

454.50 

99B 

ASTR100 

FR 

In-State 

12 

$2,172.00 

25.00% 

543.00 

99B 

ASTR100 

FR 

In-State 

12 

$2,172.00 

25.00% 

543.00 

99B 

ASTR100 

FR 

In-State 

12 

$2,172.00 

25.00% 

543.00 

99B 

ASTR100 

FR 

In-State 

13 

$2,172.00 

23.08% 

501.23 

99B 

ASTR100 

FR 

In-State 

18 

$2,353.00 

16.67% 

392.17 

99B 

ASTR100 

FR 

In-State 

11 

$1,991.00 

27.27% 

543.00 

99B 

ASTR100 

FR 

Out-of-State 

11 

$5,731.00 

27.27% 

1563.00 

99B 

ASTR100 

FR 

Out-of- State 

12 

$6,252.00 

25.00% 

1563.00 


Financial Aid 

In calculating the amount of revenue generated by an 
online course’s enrollment, the results need to be 
adjusted based on financial aid. Just as the 
institution’s course file is used to document the total 
number of classes taken by students in the course, 
financial aid extracts are used for this part of the 
model (if available). However, these data files are 
complex and easy to misinterpret. 

Financial aid data are used to offset the expected 
tuition and fees charge. This is the net of the tuition 
and fees charges after any internal tuition discounting 
or waiver that may be in place. For most purposes, it 


is adequate to estimate the percent of enrollment by 
course level with waivers and with tuition discounts. 
The average tuition discount per student level or 
course level may be calculated by the financial aid 
office, perhaps as part of routine reports for 
admissions guides or the institutional fact book. 
These discounts are often labeled as institutional 
scholarships or grants. 

Revenue Stream Calculations 

From these raw course and tuition and fees data by 
student, it is possible to aggregate or group to the 
course section and calculate the total revenue stream. 


Table 4-26: Estimated Revenue Stream per Course Section 


Section 

In-State 

Tuition 

Out-of-State 

Tuition 

Total Class 
Tuition 

In-State 

Enrolled 

Out of State 
Enrolled 

% In-State 

Total 

Enrolled 

ENGL300A 

$5,593.31 

$0.00 

$5,593.31 

13 

0 

100.0% 

13 

ENGL300B 

$7,554.82 

$4,298.25 

$11,853.07 

19 

3 

86.4% 

22 









MIS 200A 

$28,030.32 

$9,321.39 

$37,351.71 

69 

7 

90.8% 

76 

MIS 200B 

$31,374.79 

$5,205.13 

$36,579.92 

77 

4 

95.1% 

81 









DESC200A 

$33,740.87 

$14,947.61 

$48,688.48 

73 

12 

85.9% 

85 

DESC200B 

$24,521.05 

$15,445.97 

$39,967.02 

50 

12 

80.6% 

62 









ASTR 100 A 

$21,677.23 

$2,571.60 

$24,248.83 

46 

2 

95.8% 

48 

ASTR 100B 

$127,597.84 

$45,706.41 

$173,304.25 

272 

35 

88.6% 

307 
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Table 4-27: Adjusted Revenue Stream based on Institutional Aid 


Section 

Total Class 
Tuition 

Total 

Enrolled 

Full-time 

Institutional. 
Aid (2%) 

Amount of 
Aid 

(3,561 avg) 

Adjusted 
Class Tuition 

ENGL300A 

$5,593.31 

13 

21 

0 

0 

$5,593.31 

ENGL300B 

$11,853.07 

22 

8 

0 

0 

$11,853.07 








MIS 200A 

$37,351.71 

76 

74 

1 

3,561 

$33,790.71 

MIS 200B 

$36,579.92 

81 

72 

1 

3,561 

$33,018.92 








DESC200A 

$48,688.48 

85 

80 

2 

7,122 

$41,566.48 

DESC200B 

$39,967.02 

62 

55 

1 

3,561 

$36,406.02 








ASTR 100 A 

$24,248.83 

48 

37 

1 

3,561 

$20,687.83 

ASTR 100B 

$173,304.25 

307 

285 

6 

21,366 

$151,938.25 


To calculate the effect of institutional aid, student enrollment data are analyzed based on the number of credit hours 
taken to document whether students are categorized as full- or part-time (for purposes of determining their eligibility 
for financial aid). The number of full-time students is totaled for each course section. Data reported by institutions to 
the NCES for 1998-99 financial aid awards was used to get the percent of students receiving institutional aid 

Two more cost drivers for the model are the percent of students receiving institutional grants and scholarships and 
the average award amount at this institution (2%) and the average amount of aid they received ($3,561). This figure 
was applied to the hill-time student number to estimate the number of students who would probably have received 
institutional aid. This financial aid per section is subtracted from the total tuition estimate obtained above to get an 
adjusted revenue stream. Table 4-27 documents these calculations. 

Differential Tuition Rates and Revenue Streams 

There has been some timely discussion about differential tuition rates for online and site-based classes. Why should 
online students help pay for services they do not use? Also, if online courses are often more expensive than site- 
based classes to develop, should not this cost be passed on to students? 

This issue is muddied because the few studies that have been done about online versus site-based courses suggest 
that it is traditional age campus students who often take online classes to supplement regular classes. Some 
institutions claim that they do not want to attract outside, part-time students with their online courses, but simply to 
serve their existing students in different and more efficient and effective ways. 

For this reason, a true discussion of adjusted revenue stream and the net cost of online courses must somehow 
account for the difference in expenditures serving students who take only online courses versus those who use 
campus services. The indirect cost approach used in this GMU model effectively passes on the cost of libraries and 
student services and technology to all students based upon student credit hour consumption. Yet online students do 
not use these services. They do, however, use other services more intensely, such as online registration and all of the 
support costs of online technology. Note that direct costs of the online technology, if known, are included in the 
class cost; but only if there is an appropriate way to amortize, pro-rate, and allocate these costs across some useful 
measure of consumption. An example would be the number of postings per class to a threaded discussion group, 
divided somehow by the number of postings and threads for all classes using the software during a period of time. 

To account for these differences, users must be informed about the separate technology costs for every unit serving 
online students. While these studies have been done and there is much interest in understanding the foil cost of 
online courses, this is outside of the purpose of this chapter, which is 
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Table 4-28: Performance Measure Results per Course Section 



Online 

ENGL300A 

Classroom 

ENGL300B 

Online 

MIS200A 

Classroom 

MIS200B 

Online 

DESC200A 

Classroom 

DESC200B 

Online 

ASTR100A 

Classroom 

ASTR100B 

Total Expenditures 

$16,000.81 

$15,063.68 

$72,886.33 

$61,131.78 

$58,215.96 

$45,781.55 

$82,413.52 

$166,277.81 

Total Revenues 

5,593.31 

11,853.07 

33,790.71 

33,018.92 

41,566.48 

36,406.02 

20,687.83 

151,938.25 

Net Cost per Section 

10,407.50 

3,210.61 

39,095.62 

28,112.86 

16,649.48 

9,375.53 

61,725.69 

14,339.56 

Course Credit Hours 

39 

66 

228 

243 

255 

186 

144 

918 

Cost per Course 
Credit Hour 

410.28 

228.24 

319.68 

251.57 

228.30 

246.14 

572.32 

181.13 

Net Cost per Course 
Credit Hour 

$266.86 

$48.65 

$171.47 

$115.69 

$65.29 

$50.41 

$428.65 

$15.62 


reporting on the GMU implementation of a modified 
Flashlight cost model. A true activity based costing 
approach of non-academic units should be done in 
this area. Indiana University’s Responsibility- 
Centered Management (RCM) approach is an 
excellent example of how to break apart the usual 
focus on cost centers to look at the cost of an activity 
across different administrative and academic units. 

10. Summarize the results 

This final step brings the expenditure and revenue 
components of the model together to calculate the 
overall cost of course sections using the performance 
measures. Table 4-28 depicts the performance 
measure results of the completed study. 

Discussion 

The results suggest, first of all, that it is possible to 
develop this type of complex model with data 
available within the existing administrative 
information systems of most universities. 

Second, in terms of total expenditures, all of the pairs 
of sections are relatively in the same range except 
ASTR, which has high overhead per course credit 
hour but also high revenue. 

Net costs per section are noticeably higher for online 
courses. The ratio is 3:1 for ENGL and 4:1 for 
ASTR. Total net costs per section vary widely based 
on the institutional overhead cost, which increases 
with course credit hours. Net costs per course credit 
hour, which evenly distributes institutional overhead, 
vary widely, from $15.62 for a traditional section of 
astronomy to $428.65 for the online section of the 
same course. The MIS and DESC net costs are 
somewhat similar between the traditional and online 
sections, but the online version of the English course 


is five times the traditional. Why is there so much 

variation? 

□ There is a benefit to offering online courses in 
departments and divisions that already generate 
extensive course sections. It is the allocation of 
division overhead charges that make the online 
astronomy section expensive to offer. If the same 
section were offered within the department of 
Physics and Astronomy, housed in the College of 
Arts and Sciences, the costs would be much 
lower, even with the same faculty compensation 
expenditures. Perhaps it should have been costed 
within this department, but it was the more 
expensive division where it originated. 

□ There are significant startup costs in personnel. 
Even though amortized over the course of six to 
twelve course offerings, the online course will 
bear this additional expenditure. Hopefully, 
content development is not an ongoing process. 
At some point in time, online content may begin 
to pay for itself. This would show up in the 
model as decreased faculty workload in teaching 
the section and decreased need for additional 
personnel costs. 

□ Full-time faculty, most of them tenured and at 
least one at the full professor level, teaches all of 
the online courses. This ensures the quality of 
the online offering, but increases costs. The 
traditional sections of ENGL and MIS are much 
more expensive if full-time faculty are used 
instead of part-time. This brings up the necessity 
of comparing apples to apples and oranges to 
oranges. Only the DESC pair of sections 
includes the same use of full-time faculty and the 
same overhead structure. ENGL and MIS have 
different types of faculty in the comparison 
sections. ASTR crosses divisions and 
departments in overhead. In hindsight, costs 
should be modeled only for course sections that 
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share the same attributes and differ only in their 
use of the Internet for delivery. 

□ Space is a factor in cost-savings that is realized 
with the online courses, though less so when 
these sections still meet occasionally on campus. 
This factor, too, varies by course. The section 
with the highest space costs, ASTR100B, met in 
a room that had fewer classes meeting in it. This 
higher utilization rate translated into higher 
costs. Calculation of utilization rates, though 
time consuming and difficult to do, is possible 
with existing course meeting files obtainable 
from the registrar’s or institutional research 
office. Winston’s concept of opportunity cost is a 
reasonable method for estimating the true cost of 
classrooms and takes into account depreciation 
and maintenance (Kirshstein et al, 1997; 
Winston, 2000) 

Tweaks of the Cost Drivers 

If one were to use a model of this type to estimate 
whether online and traditional courses have 
fundamentally different costs, it would be essential to 
select pairs of online and traditional sections that 
have the same faculty compensation rate; faculty 
workload; departmental, division, and institutional 
overhead; headcount enrollment and course credit 
hours; and use of office space. Then the only 
difference would be in the use of other personnel and 
direct OTPS costs to develop the online course, the 
use of classroom space, and increased use of 
computing equipment and software. 

While the traditional section has additional space 
costs, it does not have the development costs. Over 
time, as development costs for the online course 
decrease, it becomes proportionally less expensive 
than the traditional section. 

A similar scenario could be created in order to 
estimate the effects of increased enrollment on costs. 
With everything else being equal, the online section 
would generate more revenue without additional 
space demands; in contrast, the traditional class is 
bound to available classroom scheduling and faculty 
workload constraints. Given that so many factors 
may not be equal in a particular costing model, 
planners need to construct worksheets that will allow 
them to tweak different cost drivers and build these 
types of scenarios. 


document and question assumptions. Some of the 
most important by-products of this process are 
information about the changing nature of faculty 
roles in online teaching and a better understanding of 
cost drivers and how these may be used to ensure 
cost-effective uses of technology. 

Clearly, one of the greatest hurdles for planners is in 
gathering the data they need about faculty workload, 
administrative cost sharing, and other hidden or 
indirect costs such as computing support. For the 
purposes of the study described in this paper, it was 
impossible to collect activity-based costing data 
within the constraints of the project timeline and 
scope. Adequate assumptions about faculty workload 
for instruction and for the individual course section 
may be made, but the argument for better data is 
compelling. 

What is most critical to helping institutions prepare 
for cyber-colleges and web-based courses is a vision 
of what is possible with resource planning models of 
this type. By following the steps of the Flashlight 
Handbook, planners are better able to address the 
competing tensions and priorities of complex 
academic issues. Planners and institutional 
researchers alike do their institutions and themselves 
a disservice if they neglect to use the tools and data at 
hand to help make decisions about the future of 
online learning. 


Perhaps the most valuable use of models lies in 
forcing planners and institutional researchers to 
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CASE STUDY 5: 

The ‘Learning to Teach With Technology’ Studio 
Mainstreaming Cost Analysis into the 
Development and Evaluation of Distance Education Products 

Susan A. Tucker, Evaluation & Development Associates and 
Jamie R. Kirkley, Indiana University 

Abstract: This chapter examines issues with mainstreaming cost analysis into the development 
and evaluation of a distance education product. The Learning to Teach with Technology Studio 2 
(LTTS) is an online professional development system designed to help teachers learn to 
integrate technology to support inquiry and problem solving. A three-year case study of the 
project ’s cost model is presented along with barriers related to mainstreaming the cost model 
system into the project’s development and evaluation cycles. Issues related to understanding the 
cost model system as an ever-evolving process are also addressed. 


2 Development of the LTTS is funded, in part, through the Department of Education’s Fund for the Improvement of Post Secondary Education 
(FIPSE) Learning Anytime Anywhere Program (LAAP) (USDE Grant P339B990 108-01). For more information, see http://ltts.org. 
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Using Cost Analysis as A Tool For 
Guiding an Evolving Project 

For distance education professionals, using a cost 
model system can be an extremely effective tool for 
documenting the development and delivery costs of 
courses or entire distance education projects. As the 
following case study shows, cost analysis can also 
contribute to decision-making about online module 
development. 

Cost models have traditionally been used to make a 
summative statement of costs. But waiting until the 
end of a project does not allow the outcomes of the 
model to feed into the project’s management 
decisions or to facilitate timely course development 
By building the cost model system into the ongoing 
life of the project, it can be used as a formative 
evaluation tool to examine issues and questions about 
costs, resources, and time investments as the project 
is being developed. In essence, this mainstreaming 
dramatically increases the potency of the cost model 
as a decision-making tool and enables staff and 
evaluators to monitor and reflect on the ongoing 
changes to the project as well as to the cost model 
system. 

This chapter describes the changes made to one cost 
model system over time as well as the methods used 
to mainstream it into the decision-making processes 
of the project. The project described is a rather 
unusual distance education program focusing on 
providing online professional development modules 
in technology integration for in-service and pre- 
service teachers. The goal of this chapter is to help 
those who are developing cost model systems 
understand specific issues with regard to 
mainstreaming the system, such as barriers to 
gathering data, as well as how to deal with changes in 
die model as the project itself evolves over time. 

About the Project and Its Evolution Over 
Time 

In 1999, the National Center for Education Statistics 
found that 80% of pre K-12 teachers do not feel 
prepared to integrate technology into their curricula. 
One of the reasons cited is a lack of relevant courses 
for both teachers and students in teacher education 
programs. To meet this need, an Indiana University 
team has been collaborating with several partners to 
develop an online professional development system 
for teachers: the Learning to Teach with Technology 
Studio (LTTS). A five-year grant funded by the U.S. 


Department of Education’s Fund for the 

Improvement of Post Secondary Education (FIPSE) 
was awarded to Indiana University in August 1999 to 
develop this system. 

The goal of LTTS is to provide web-based 
professional development to help K-12 teachers learn 
to use technology to: 

• Support student inquiry and problem solving; 

• Develop learner-centered curriculum and 

teaching strategies; 

• Create technology-based projects for their 
classrooms; and 

• Teach with technology in a manner consistent 
with professional standards. 

LTTS addresses this overarching goal through three 
main means: 

1 . Offering focused, online instructional modules to 
teachers that focus on using technology to 
support student inquiry 

2. Providing a support system for teachers to 
develop additional modules; and 

3. Using an e-commerce system to scale and sustain 
the model. 

The goal of each LTTS module is to help a teacher 
design and produce a technology-based project, such 
as a teaching unit or project plan, for his or her own 
course. The modules are valuable for more than just 
their own content: a core premise of the project is 
that the process of developing such modules can also 
help teachers reflect on their own teaching practices 
related to inquiry and technology and share their 
expertise with other teachers. 

The modules are titled to address specific teacher 
questions such as, “How can I design a webquest to 
use in my classroom?” and “How can I use learning 
stations to integrate technology in my classroom?” 
(See Figure 5-1.) 

K-12 teachers as well as graduate students with K-12 
teaching experience develop LTTS modules. 
Teachers use LTTS development guidelines, 
templates, and other materials such as web resources 
on their module topic. Each module developers work 
with a LTTS mentor who helps him or her manage 
the development process. Figure 5-1 illustrates a 
sample LTTS module’s structure. 
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Figure 5-1: Example of an LTTS online module 
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The original conception of the LTTS project was to 
provide a learning anytime, anywhere environment 
that would be highly scalable and sustainable. This 
would be done by high quality learning modules for 
individual learners in an online learning environment 
with no facilitators or mentors. Teachers would pay a 
tuition fee that would go back to LTTS with a portion 
to be distributed back to the module developer as 
royalties. Developers of modules were to be K-12 
teachers with expertise in technology integration. As 
part of the LTTS system, teachers could develop 
modules to add back to the system in exchange for 
royalties. It was envisioned that they would use an 
online support system called the Design Strategy 
Environment, which consisted of templates and 
design documents as well as minimal human support. 
Hence, human interaction in the system was initially 
minimized in an effort to quickly scale and sustain 
LTTS with little administrative overhead. 

However, after almost three years of developing the 
project, supporting human interaction through 
facilitation and mentoring has become a critical part 
of the online learning process as well as the module 
development process. We found that teachers want 
and need more support and feedback than originally 
anticipated. Rather than focusing on just individual 


teachers, as we had originally intended, we are now 
designing a mentoring system (both virtual and 
human) to support groups of schools and districts and 
teacher preparation programs that wish to use LTTS 
as a professional development tool. Thus, the project 
has evolved as a result of our findings from working 
with teachers, schools, and universities. For example, 
at the end of the project’s second year, the module 
development process was re-conceptualized to 
include face-to-face workshops, online mentoring, 
and a modified online support system for the 
teacher-developers by mentors who are graduate 
students experienced in using the system and most 
have a K-12 teaching background as well. 

In projects such as ours, developers constantly 
change as they become more aware of audience 
needs. Also, their skills as developers improve over 
time as they develop more modules, as they make 
adjustments to the project as it is being developed, 
and as they respond to new markets for the product 
being developed. 

As strategies and developers change, cost analysis 
needs to change with them, and, in that process, help 
guide the program’s evolution. 
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Setting Up the Activity-Based Cost Model System 

From the start of the project in August 1999, the 
project manager used the Flashlight approach to 
developing activity-based cost models by using an 
early version (Ehrmann, Lovrinic, and Banta, 1999) 
of the Flashlight Cost Analysis Handbook (Ehrmann 
and Milam, 1999). 

This activity-based cost model has enabled us to 
describe the process where resources are consumed to 
produce specific products or outputs. The outputs, 
which are created as a result of the activities, utilize a 
variety of resources. Fortunately, one of the authors 
of the original cost model document, Joseph 
Lovrinic, who worked as a management advisor for 
the Vice President and Chief Financial Officer of 
Indiana University, served as a mentor to the 
project’s manager, Jamie Kirkley. Lovrinic helped 
Kirkley set up and maintain the cost model system. 
They met semi-annually to address issues specific to 
developing the model within the LTTS project. 

LTTS used an activity based cost model approach for 
several reasons: 

• To determine development costs for various 
aspects of the LTTS system; 

• To allocate resources effectively and contain 
development costs; and 

• To help evaluate the development and use of the 
project. 

Using the Flashlight approach as detailed below, the 
project manager developed the cost model system 
used by LTTS, with input from the evaluator and 
project staff. The staff was always interested and 
helpful, although most found it a chore to track then- 
time so systematically. Within an academic 
environment, tracking time on specific aspects of a 
project is not a cultural norm. 

The LTTS project has goals similar to many 
development projects. Consider the following list of 
first year goals. Specifically, our cost model was 
supposed to describe the resources needed to do each 
of these sets of activities: 

• establish the project team and the 
communication methods; 

• establish the LTTS homepage, which would 
serve as a gateway to the online learning 
modules as well as the support system for 
module developers; 

• hold an Advisory Board meeting to review our 
plans and initial work and offer guidance on 
strategies; 


• establish an initial strategy for tagging the 
modules to support searching and the eventual 
e-commerce environment; 

• perform ongoing user testing of the interface, the 
modules, and the design advice; 

• develop ten prototype learning modules, the 
LTTS interface, the support system for module 
developers, and the definition of those 
design/technology constraints relevant to 
achieving technological usability; 

• solicit developers to prepare additional modules 
consistent with our specifications; and 

• begin to develop the marketing strategy. 

An important part of setting up the cost model was to 
discuss it with the project team in order to get team 
buy-in into the cost model system. Buy-in is critical 
because it is important for staff members to 
understand how data would be collected and used. It 
also helps them understand that gathering data for the 
cost model is not a meaningless ritual — it saves them 
time and effort in the long run. Making sure the staff 
knows how data will be collected and used is a 
critical part of sustaining a cost model system. 

Having staff buy-in and input on issues has been 
helping improve the development of and use of the 
cost model. Specifically, the LTTS team was asked to 
describe the types of activities they would be 
performing; these descriptions helped us develop the 
cost model. We learned quickly that it takes time for 
a staff to learn enough to help build and use a 
sophisticated cost model. We initiated a system that 
had many components that overwhelmed our staff, 
particularly the graduate students. So we decided to 
proceed cautiously with the pace and scale of our 
requests for data, hoping that we would be able to 
increase the complexity and power of the system over 
the course of five years. So far we have been 
moderately successful. As the staff has learned and 
become more committed, we have been able to 
improve our model. The project manager usually 
goes through a review of the cost model each time 
there are staff additions, major changes in activities, 
or new outputs added. Working with the external 
evaluator, the cost model’s effectiveness has been 
reviewed and indicators expanded each year and 
some areas deleted. 
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Expect changes: as the project changes so does the 
cost model 

As we tell the story of the development and use of the 
cost model for LTTS, it is important to note several 
changes in the project, which also facilitated changes 
in the cost model system . 

First, as described above, the LTTS was originally 
conceived to be a pure “learning anytime anywhere” 
system that teachers could use individually with little 
interaction with an instructor or peers. Because such 
a system would be less expensive to operate and 
because costs per user would decrease rapidly as the 
number of users increased, it ought to be easier to 
make self-sustaining. However, in the first two years 
of the development cycle, we found that human 
interaction and support were critical for helping 
teachers work through the issues of developing 
inquiry based modules for adult learners. In part that 
was because teachers were not only learning about 
building modules; they were also learning the theory 
and methods of a new model of instruction, inquiry- 
based education. Also, we had underestimated the 
need to train teachers in how to teach their peers. We 
had ignored the fact that teachers most often develop 
curriculum for preK-12 students, not for other 
teachers. Therefore, LTTS module development 
mentors had to spend much more time than originally 
budgeted in order to orient new teacher recruits as 
well as continue to help teachers learn these three key 
skills throughout the development process: teaching 
peers, teaching by inquiry, and building online 
modules. For these and other reasons, formative 
evaluation studies showed clearly that our teacher- 
learners expected feedback from mentors along the 
way (preferably at each step), even if the teacher- 
learners were not taking the module for credit. So the 
system was redesigned to provide a human mentor 
(rather than an online support system) to provide 
appropriate and timely feedback on modules 
developed as part of the learning experience. 
Obviously, the costs of this change of project 
direction have been considerable. We are hoping that 
given this shift, we can develop a trainer of trainers 
dissemination model to defray the cost of training 
mentors, both locally and at sites distant from ILF. 

Second, major changes in project partners over the 
first three years of the grant resulted in substantial 
changes in the project’s outputs as well as 
development strategies. In particular, the addition of 
PBS TeacherLine as a project partner added two 
more product outputs — 12 modules to be licensed 
specifically to PBS and video vignettes to be 
developed and packaged with those 12 modules. 
Furthermore, in the second year of the grant, Indiana 


University took over the development of the online 
guidance system for supporting module development 
from another partner. As the partner and development 
responsibilities shifted, the roles of the staff also 
changed. All these changes in outputs and activities 
required making more changes in the project’s cost 
model. 

A third impact on the cost model resulted from a 
decision of the LTTS management to decrease the 
number of modules to be produced in order to put 
more emphasis on teacher outreach and professional 
development. Specifically, LTTS decided to provide 
professional development services and tools for a 
small group of schools. This generated yet another 
output - LTTS-based professional development 
programs. LTTS is currently recruiting several 
schools to participate in a two-year professional 
development effort to use and develop modules in 
order to meet goals for classroom based technology 
integration. These changes had to be reflected in the 
cost model as well as in the gathering and analysis of 
data. 

Developing the LTTS Cost Model System 

Every year, our cost model development (and 
redevelopment) has followed the same seven steps 
described in The Flashlight Cost Analysis Handbook : 

Step 1 . Identify the business concerns. 

Step 2. Identify the outputs. 

Step 3. Identify the activities. 

Step 4. Identify the organizational units that 
participate in the production of 
project outputs. 

Step 5. Identify the resources these units consume to 
perform their activities. 

Step 6. Calculate the costs for these activities. 

Step 7: Tally the costs of all activities to arrive at 
overall costs. 

To demonstrate how our cost model system evolved 
each year, this section presents a comparison of the 
first three years. After we discuss the system’s seven 
steps, we will discuss specific challenges related to 
mainstreaming the cost model into the project’s 
development and evaluation processes. 

Step 1: Identify the business concerns 

The first step in building our cost model involved 
identifying business concerns. Identifying business 
issues as we began building our model helped to 
make project development decisions. LTTS business 
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concerns for the shifted during the project’s first 
three years as depicted in Table 5-1. Note that with 
the passage of time, the business concerns began to 


shift from development to e-commerce and 
marketing of the program. 


Table 5-1: Shifting business concerns across each year of the LTTS project 


Project Shifts 

Year One Concerns 

Year Two Concerns 

Year Three 

1) Targeted users of modules: 
from teachers and 
technology coordinators to 
teachers and graduate 
students in year 3. 

2) Project changed with regard 
to how much online 
facilitation would be 
provided. In year 3, the 
LTTS moved towards 
developing mechanisms 
needed to support 
facilitators in the system. 

Instructional modules 
developed for K-12 
teachers and technoloev 
coordinators for 
professional 
development in 
technology integration 
issues; modules 
designed to be learning 
anytime anywhere and 
to require no 
interaction. 

Instructional modules 
developed for K- 1 2 
teachers (mainlv 
inservice) for professional 
development in 
technology integration 
issues; modules are 
designed to be learning 
anytime anywhere and 
require no interaction with 
a facilitator. 

Instructional modules 
developed for K-12 teachers 
(mainly inservice) for 
professional development in 
technology integration 
issues; modules are learning 
anytime anywhere but 
require support and 
interaction from online 
facilitator. 

Note the shift from a fully 
online support system to a 
combination of online and 
face-to-face. 

Online design strateev 
environment for 

Face-to-face module 
development svstem with 

Mixed model of module 
development including both 

teachers who want to 
create these modules. 
This system was to 
provide an online 
performance support 
system including 
guidance, strategies, 
and tips. 

some online mentoring 
and mostlv paper-based 
templates to support 
developers. 

face-to-face mentoring and 
online guidance documents 
and templates to support 
developers as well as their 
mentors. 

Originally we intended to 
simply to sell modules (e- 
commerce). After a while we 
realized we would also be 
using them for distance 
learning, so we also 
incorporated a course delivery 
system including personal 
learning tools (discussion 
forum, etc.) 

E-commerce svstem for 
creating a self- 
sustaining environment. 

We began development 
of a main homepage so 
users would have a 
gateway to the courses. 

(Note: During the first 
year, we could not even 
define the parts of the e- 
commerce system. So 
this was not an 
appropriate output for 
this time period.) 

LTTS learning svstem for 
managing course deliverv 
and learning. 

We began developing a 
learning system to support 
delivery of the courses 
because commercial 
systems were not adequate 
for our goals. Our system 
included a discussion 
forum, internal messenger 
system, notepad, and a 
personalized and password 
protected “My Desk” area 
to track modules in 
progress and modules 
completed for each 
learner. 

We also refined the main 
portal to include a module 
tour, catalog, and news 
and events 

LTTS svstem for managing 
course deliverv, learning, 
and communication with e- 
commerce components to 
support online purchases of 
modules. 

This now involves a system 
for delivering the courses, 
enabling communication 
among learners, supporting 
facilitators, and supporting 
the administrative and 
facilitator functions of 
LTTS. The e-commerce 
system consists of a module 
catalog, enrollment system, 
and a method for 
purchasing modules online. 

(Note: The process for 
awarding royalties is still 
under development and is 
still undefined.) 
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Step 2: Identify the outputs 

Next, we identified the products to be developed. Our 
description of products shifted over the three years of 
project work, as Table 5-2 illustrates. Notice that the 
shifting names of outputs provide clues to the 
changes in directions of the project. For example, the 
name “Learner’s Studio” was chosen in year 1 to 
represent a place where learners could create things 
(in keeping with the focus of LTTS on developing a 
set of modules to use in the classroom). The later 
name “Learning Module Environment” reflects a 
more holistic view of the process of module 
development, including such tools as MyDesk. The 
final name, the “Learning Module System,” 
represents the modules, the delivery system, and all 
the tools (e.g., discussion forum, notepad) that go 
with it. 


As the project changes over time, it is natural for 
outputs to change. For example, while not envisioned 
during the first year of the project, additional funding 
from PBS was generated at the end of the first year to 
produce video vignettes, which added another 
product. In year three, LTTS began offering modules 
to learners online, so Learning Module Usage was 
added as an output. Costs associated with usage of 
the modules were differentiated from development 
because we wanted to address the question of how 
much it would cost to maintain facilitators and 
administration to deliver the modules to teachers. In 
the beginning, LTTS staff reluctantly understood the 
need for cost modeling, at least in theory. As time 
went by, however, staff began taking part in a 
proactive process that asked them for suggestions and 
refinements in the description of outputs. 


Table 5-2: Outputs occurring during the first three years of the grant 


Year One Outputs 

Year Two Outputs 

Year Three Outputs 

Administration 

Administration 

Administration 

Learner’s Studio 

Learning Module Development 

Learning Module System 

Developer’s Studio 

Design Strategy Environment 

Developer Support System 

LTTS Portal 

Main LTTS homepage 

PBS Modules /Videos 

Professional Development 

Learning Module Usage 
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Step 3: Identify the activities 

The third step in implementing the Flashlight cost 
modeling strategy was to identify the activities that 
go into developing LTTS outputs. Activity analysis 
actually has two parts: 

1) Identifying the individual steps needed to 
produce an output (step 3, described here); and 

2) Identifying the organizational units and people 
involved in those steps (step 4, described in the 
next section). 

To help gather this data, we asked each employee to 
complete an ‘activity tracker sheet’ every week in 
order to document where that employee’s time was 
invested. At the start of the project, staff kept track of 
this data in a word processing document; the project 
manager collated the data manually. Now (in year 
three), each staff member completes a timesheet 
using Excel once a month rather than once a week. 
The project manager copies the data from each 
timesheet into her master spreadsheet, which 
eliminates all data entry and enables examination of 
costs during development cycle rather than only at 
end of it. 


While the theory of cost modeling was easy to grasp 
by project management and evaluation team, the 
nuances required for implementation were not. While 
not readily apparent when we first adopted the cost 
model, there were several challenges with developing 
activities for a cost model: 

• knowing which questions you will need to ask 
and collecting data that will enable you to 
answer those questions; 

• collecting data at the proper level of detail; and 

• collecting data in a format that will be useful. 

Specific examples of this are discussed in more depth 
in the section titled Barriers and Challenges to 
Mainstreaming. 

Activity tracking sheets became increasingly 
sophisticated as staff members grew accustomed to 
the cost model and became open to keeping records 
of more detailed tasks (e.g., research versus 
usability). Tables 5-3 and 5-4 show how we have 
evolved in just the first two years alone. However, we 
have learned that this is a gradual process. First, we 
had to gain a better understanding of how the project 
was developing overall. Second, it took time for the 
staff and project manager to learn to manage the 
process. In short, we could not have accelerated to 
the Year Three Activity Sheet during the first year. 
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Table 5-3: Year One LTTS Cost Model Activity Sheet 


Name Week of 

Note: Meetings should not be counted in administrative time. 
Please break up meeting times by topics covered below. 


Administrative 

Hours 

Gen admin (email, clerical) 


Planning & mgmt 


Computer/server maintenance 


Publicity/presentations 


Budgeting/cost model 


Evaluation 


Professional Development 


Advisory board 


Paid time off 


Other System flowchart 



LTTS Portal 

Hours 

Develop content 


Design interface 


Develop portal (html, java) 


Design graphics 


Programming 


Usability/Review 


Research 


Develop multimedia 


Other: 



Learner’s Studio 

Hours 

Design instruction 


Design interface 


Develop modules (html, java) 


Design graphics 


Programming 


Usability/Review usability/plan 


Research 


Develop multimedia 


Other: 



Design Strategy Environment 

Hours 

Design instruction 


Design interface 


Develop DSE (html, java) 


Design graphics 


Programming 


Usability/Review interview notes 


Research 


Develop multimedia 


Other: 



Note: (This is where you put notes on tasks in which you do not know how to categorize.) 
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Compare Table 5-4 with Table 5-3 to see how the cost model changed, reflecting the changing concerns, outputs, 
and activities. We have noted in bold those activities that are common to the first three years of the grant order to be 
able to compare how the system changed and became more complex. 

Table 5-4: Year Two LTTS Cost Model Activity Sheet 


StaffName: 

Hours 

Time Category 


1. Output: LTTS Portal 


1.1 Develop content 


1.2 Communicate with LTTS staff 


1.3 Plan and manage 


1.4 Design interface 


1.5 Develop portal (html, java) 


1.6 Design graphics 


1.7 Programming 


1.8 Conduct usability/Review 


1.9 Conduct research 


1.10 Develop video/multimedia 


1.11 Develop IMS meta data 


1.12 Develop design documentation 


1.13 Other: 


1.14 Other: 


1.15 Other: 


SUBTOTAL 



2. Output: Learning Center 


2.1 Design instruction 


2.2 Edit modules 


2.3 Tutor LTTS developers 


2.4 Develop/conduct quality control of modules 


2.5 Design interface 


2.6 Develop Learning Center content 


2.7 Develop modules (html, java) 


2.8 Design graphics 


2.9 Program components of Learning Center 


2.10 Conduct usability/Review 


2.11 Develop video/multimedia 


2.12 Conduct research 


2.13 Develop design documentation 


2.14 Other: Programming 


2.15 Other: 


2.16 Other: 


SUBTOTAL 
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3. Output: Production Center 


3.1 Design instruction 


3.2 Design interface 


3.3 Develop DSE (html, java) 


3.4 Design graphics 


3.5 Programming 


3.6 Usability/Review interview notes 


3.7 Research 


3.8 Develop multimedia 


3.9 Other: 


3.10 Other: 


3.11 Other: 


SUBTOTAL 



4. Output: Administrative 


4.1 Gen admin (clerical) 


4.2 Planning & mgmt 


4.3 Computer/server maintenance 


4.4 Publicity/presentations 


4.5 Budgeting/cost model 


4.6 Project evaluation 

—collect data & analyze & review 
—prepare annual reports 


4.7 Professional development 


4.8 Advisory board meeting/communication 


4.9 Partnership activities 


4.10 Marketing/business development 


4.11 Paid time off 


TBD: prepare developer training materials, 

edit training prototypes, 


TBD: recruitment, enroll students/developers, 
confirm registration, monitor retention 


4.12 Other: flyer development 


4.13 Other: training 


4.14 Other: 


SUBTOTAL 


TOTAL 


5. Comments 






Table 5-5. Organizational units that participate in the production of outputs 



Activities & tasks 


Director 

Clerical 

Developer 

Programmer 

Other 


















Step 4: Identify the organizational units that 
participate in the production of outputs 

Next, we identified the organizational units (and the 
people in those units) who perform these activities 
and tasks in order to produce our outputs. We used 
Excel spreadsheets with rows listing the activities and 
tasks identified and columns identifying who 
performed these activities. The result was a two- 
dimensional matrix such as in Table 5-5, showing 
where individuals or offices took part in the same 
activities and tasks. When you create your own 
model, you can expect that these “units” will shift 
each year as staff and partners change. 

In order to create the cost model, we had to persuade 
specific partner groups to participate. University of 
Colorado at Denver and University of Georgia both 
had to participate in the cost model because they 
were helping develop various aspects of the project. 
Therefore, their staff also completed activity sheets. 
However, for organizations that did primarily one 
activity (such as IMS Global Learning Consortium, 
Inc., which developed the meta-tag system for our 
modules), activity sheets were not collected. At the 
end of the year, the overall cost for these 
organizations was calculated into the cost model 
system as a development cost. For example, we were 
interested only in the overall cost of meta-tags and 
not with costing specific activities for those 
development efforts. Because the meta-tag 
component was not a major output, it was not 
necessary to gather detailed development costs. 
Learning where we did not need detailed information 


on costs was important because it saved time in data 
collection and analysis. 

Step 5: Identify the resources these units consume 
to perform their activities 

The next step is to identify the resources these units 
and individuals use to produce outputs. Examples of 
resources include: 

• Compensation such as salary, retirement 
contributions, and benefits; 

• Expenses like staff consume such resources as 
supplies, room space, phones, travel, lodging, 
equipment purchases or rentals; 

• Direct allocations including both compensation 
and expenses; and 

• Hidden costs such as building and equipment 
depreciation and the cost of legal services 
provided by university at no cost. 

For example, these costs might be calculated to 
include the indirect costs taken by Indiana University 
to provide building space, office supplies, 
administrative support, and legal services for 
developing module developer contracts. 

Step 6: Calculate costs for these activities 

After identifying business concerns, outputs, 
activities, and resources and determining the metrics 
by which to quantify them, step 6 uses these data to 
calculate the costs of the various activities and tasks 
involved in producing the outputs. 
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Table 5-6. Costs born by each organizational unit 


Cost area 

List for each organization involved 

A 

B 

C 

D 

E 

Payroll 






Benefits 






Supplies 






Equipment 






Xeroxing 






Postage 






Total 







The Economic Model is a flexible strategy and we 
needed the flexibility. A more typical approach to 
costs relies on asking how many hours or days a 
week each person works on a project, and using that 
to calculate a fraction of their total salary to be 
allocated to this set of activities. But in our project, 
production relies on teams of teachers and graduate 
students who do not work full-time or even standard 
hours. It made more sense for us to work from logs of 
actual time spent on the project. 

Step 7: Tally the costs of all activities to arrive at 
your overall costs 

Each time we go through Steps 1 through 6, we 
create matrices that separately analyze compensation, 
expenses, direct allocation, and hidden costs for 
selected activities and tasks. To arrive at our overall 
costs, we then tally the costs derived from these 
individual matrices. This involves putting all the 
pieces together - each staff member’s activities, each 
of the resources consumed, and each organizational 
unit’s overall picture. We also add in indirect costs. 
When all costs have been calculated, the total should 
equal the expenditures for the year. 

Special Challenges with Calculating Costs 

The LTTS project has encountered two specific 
challenges with regard to calculating costs: 
accounting for overtime and accounting for ‘in-kind’ 
contributions. 

Technically, since we are not paying for this 
overtime, we can treat it as in-kind costs. However, 
instructing staff members on how to document these 
activities has proven challenging. For example, if the 
project manager worked ten hours of overtime, which 
activities should she list as cost versus in-kind? 
Capturing these costs is critical because these 
unaccounted activities can have a real impact on the 
overall development picture. At this point, we have 


not addressed this issue very well and continue to 
struggle with how to collect and use this data. 

The second issue relates to in-kind costs. Since LTTS 
is a federally funded project, there must be a dollar of 
in-kind match for each dollar received from the US 
Department of Education. This amounts to quite a bit 
of in-kind contributions. For example, each advisory 
board member contributes one to three days of time 
each year to review specific products. Several 
partners contribute computer equipment and software 
as in kind contributions. While we have documented 
in- kind costs each year for purposes of reporting to 
our funding agency, FIPSE, it has taken us three 
years to incorporate these costs into the cost model. 
While these in- kind activities do not have to be 
funded from our grant, adding them to the cost model 
helps us better understand the whole development 
picture. 

Barriers and Challenges to Mainstreaming 
the Cost Model System into the LTTS 
Project 

Over the last three years, we had to meet challenges 
to the mainstreaming of cost modeling into the daily 
life of the project. Some of these challenges may well 
reflect difficulties that other cost modeling projects 
will face, especially if the process is meant to be part 
of day-to-day decision-making. Below, we discuss 
some of the serious barriers to mainstreaming, 
including: technical issues with gathering data across 
multiple sites, using the system to ask “appropriate” 
questions and gather “appropriate” data, attending to 
the scalability and dissemination factors of the 
project, and ensuring team buy-in to an evolving 
system that becoming increasingly demanding of 
staff. 
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Technical issues with gathering data across 
multiple sites 

Consider our odyssey below: 

In the beginning of the project, each staff 
member completed a weekly activity sheet 
in Microsoft Word. The staff member would 
mail it to the project manager, who would 
input it into an Excel spreadsheet. This 
system worked fine for a few months when 
there was a total staff of four. However, as 
the project staff grew, doing each person’s 
data entry by hand became too time- 
consuming. Also, the room for error in 
having to transfer numbers from a Word 
form to an Excel spreadsheet was also a 
problem. To improve data gathering 
techniques, an individual spreadsheet was 
set up so that each staff member could tally 
his or her activities. This was then posted to 
a server for the project manager to obtain. 
However, this still took time as the project 
manager had to transfer each person’s 
numbers to a master Excel spreadsheet. 
Finally, in the third year, another 
improvement was made that seems to work 
well so far. Each person received an Excel 
spreadsheet where the entire year’s worth of 
activities can be tracked. There is a separate 
worksheet for each month, and a master 
worksheet to compile all the hours for each 
activity. So there is no data entry, cutting 
and pasting of numbers, or transferring of 
data. Each employee’s spreadsheet is 
submitted monthly and is tallied into the 
master set of cost model calculations. 
Employees still submit these spreadsheets 
monthly, and the project manager reviews 
them to see if they are on track or if new 
issues are arising due to needing new 
categories. This has been one of the best 
improvements made to our system. Follow- 
through is much easier since the hours for 
compiling and aggregating data have been 
reduced. 

The number of developers and partners continues to 
grow. While this is a distinct asset, the metaphor of 
herding cats comes to mind. Data collection needs to 
be streamlined so there is minimal overhead 
associated with data gathering. Thus, we are learning 
to avoid the transfer from text to spreadsheets and all 
the error checking required of the first year process. 
Currently, it is our goal to enter and review monthly 
so that questions can be answered during the 
development cycle and not at the end when it is too 


late to help program-improvement decisions. 
However, the project manager still finds it 
challenging to make time to do this. Having an 
automated system (e.g., online web page where data 
could be entered and then assimilated) where staff 
members could enter their time and activities into a 
database that could then be aggregated into a report 
would relieve her of the tracking duties and enable 
her to focus on simply reviewing the data to use in 
decision-making. 

Using the system to ask “appropriate” questions 
and gather “appropriate” data 

One of the most important things to understand about 
the cost model is that data is not gathered just for the 
sake of gathering data — the goal is for the data to 
help answer specific questions about the project 
development. The economic model has helped us 
become aware of significant questions that we must 
face to succeed. In the past three years we have faced 
a variety of questions that have shifted our cost 
model even more. For example: 

1 . How to build a “joint vision ” between two teams 
in two different locations. As many multi-site 
projects experience, translating the rhetoric of 
the original proposal into the reality of activities 
tests the assumptions of a grant’s designers. 
Given the highly theoretical tenets of the project 
dealing with definitions of what “inquiry 
learning” is, during the first year the two separate 
development teams discovered they had slightly 
different definitions of what “inquiry learning” 
meant, designed modules without calibration 
assuming they were using identical processes, 
and at a mid-year meeting discovered the 
dissonance. Cost model use in the formative 
phase was not fully in place the first semester of 
the project so we could not use it as a detector. 
When we expanded to another university to 
increase the scalability of our module 
production, we used what we learned from the 
first year, and put more specific outputs into the 
cost model that this university was using to 
document progress and was able to calibrate 
faster given the cost model monthly data. 

2. How to build buy-in to the cost model system 
when the project is being developed at two 
different sites? 

3. How to capture development of the e-commerce 
model as it becomes clearer over time? 

4. How do we monitor the costs of the various 
teacher developer training models (and hybrid 
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models) that are emerging (e.g., face-to-face 
versus online training or a mixed model) 

5. What are the cost components of module 
developer recruitment? 

6. How do we reduce the time it takes to put 
modules online and deal with an ever-increasing 
database of resources to manage? 

7. What are special issues of rapid prototyping — 
should and how can redesign and redevelopment 
be tracked? 

We have found, at times, that we were not collecting 
data at the level that would enable us to answer 
specific questions. For example, we tracked only the 
costing of module development in general and not the 
specific costs it took to develop each module or even 
parts of a module. Later on, we realized that having 
details about those specific costs could have helped 
us determine the most efficient development methods 
(e.g., would it be more efficient to use in-service 
teachers as developers or graduate students with 
teaching experience as developers). In year three, we 
are beginning to mainstream this increased detail into 
the project; that has helped us revise the cost model 
to include the time it takes graduate student 
developers (versus teacher developers) to complete 
each of the ten development steps. However, 
gathering this detail data continues to be challenging; 
consultants resist tracking their time spent on 
projects. 

Another challenge has been dealing with the 
appropriateness and consistency in how data is 
recorded. For example, a web developer and interface 
designer may work on the same component of the 
system but record them under different outputs. This 
confusion often relates to where boundaries are 
drawn in the development of our system. For 
example, the MyDesk feature, which helps learners 
track which modules are in progress and which are 
completed, is both part of the learning module 
environment and is also part of the LTTS portal. In 
reflection, the MyDesk system should have been 
made a separate output within the cost model system 
in order to track specific costs and to avoid 
confusion. In the meantime, the staff has to decide 
under which headings to record these activities. It 
helps to have a Notes section on the timesheet; an 
employee can indicate when they are having trouble 
putting something into the proper category. The 
project manager can then decide how to categorize 
that period of time. 

Specifically, making sure that data is systematically 
collected, analyzed, and reported is one of the critical 
issues faced by our project and any project doing a 


cost model. For the staff involved in analyzing data, 
the cost model is just one of the many things they do. 
From the evaluation perspective, it is helpful to 
require the cost model data from program onset so 
that there is an outside impetus to report on the model 
at least bi-annually for this project. 

Another issue related to gathering and aggregating 
data is determining the level of granularity at which 
data needs to be collected. For example, we are 
currently discovering that multiple levels of module 
development are an important consideration for 
future production decisions. We will use the cost 
model during the summer 2002 to collect answers to 
issues such as: 

1. On the average, how much time is required for 
developing a whole module; 

2. How much time does it take each developer to 
complete each of the 10 stages of module 
development; 

3. How much time does it take each mentor to 
critique each step that developers complete; 

4. How much time does it take for developers to 
complete their second modules compared with 
their first modules? That would give us an 
indication of the learning curve of developers as 
well a hint about their comfort level and 
motivation to participate in future development 
efforts; 

5. At what steps should mentors give feedback in 
the development process given an analysis of the 
completion rate differentials; 

6. How much does the level of complexity of 
module objectives and the type of content (e.g., 
linear topics such as explaining how to use 
Geometer Sketchpad versus non-linear topics 
such multicultural technology applications) 
affect the length of time of project completion as 
well as the expertise of developers; and 

7. How much time is required by teacher 
developers compared with graduate student 
developers? 

To gather this kind of detailed data requires making 
the cost model a regular part of the monthly project 
evaluation processes. Also, it is important to try to 
incorporate additional questions that arise as the 
project evolves. At times, additional outputs and 
activities will need to be added to the activity sheet, 
so anticipating these is helpful for making sure that 
appropriate data is gathered as the new activity 
begins. For example, the addition of the PBS video 
vignettes to this project added another output for the 
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project. This part of the project required additional 
staff, resources, and project coordination. Therefore, 
it should have been added to the cost model system 
sheet before the work began. 

Since the project has used a rapid prototyping method 
of development, it can be challenging to answer 
questions about phases of development. It is 
sometimes difficult to distinguish between versions, 
because the process of development can blur from 
one version to the next. Thus time spent on particular 
versions has proven to be difficult to capture for the 
project. However, we realize if we are really 
committed to testing how well a cost model can be 
mainstreamed into our management process, we must 
address this challenge as well. 

How to ensure team buy-in to an evolving system 
that becoming increasingly demanding of staff 

Lastly, gaining staff input and trust in the model is a 
critical part of the cost model development. Staff 
input is a critical part of helping determine the 
activities to document and questions to be asked. 
After all, it is the staff that often has questions about 
the effectiveness of specific development methods. 
We have discovered the following strategies: 

• Establish buy-in early on at multiple 
development sites; 

• Address staff concerns with use of cost model 
system; 

• Invite staff reviews of the model and solicit 
regular input into the activity sheets — unilateral 
requirements to use the model are ineffective and 
result in inaccurate data. The staff must feel like 
they are building the model and understand how 
it is constructed; 

• Demonstrate that the findings are being used to 
improve the project; 

• Early on, anticipate and deal with participant 
fears of personal tracking, which can distract 
from project focus to document costs of 
activities and outputs; and 

• Be sensitive to the semiotics that surrounds this 
effort of cost modeling and the possible loss of 


the “human factor” and creativity of staff. For 
example, we changed the term “timesheets” to 
“cost model activity sheet” to break the negative 
connotation of timesheets. 

Summary 

In this chapter, we have discussed a range of issues 
with implementing a cost model within the LTTS 
project case study. These issues all relate to the 
experience of implementing and using a cost model 
as a formative evaluation tool within a distance 
learning project. Perhaps one of the most challenging 
parts of developing a cost model is helping the staff 
reach the understanding that it is a living and 
breathing entity that changes as the project changes. 
Therefore, it is critical that the cost model system be 
mainstreamed into the decision making process of the 
project. This not only enables project administrators 
and staff to make more effective development 
decisions, it also helps staff members understand the 
administrative issues with regard to project 
development. 

With the rapid growth of distance education, the cost 
model system can be an effective system for 
monitoring costs and guiding project or course 
development. However, unless barriers are addressed 
and the cost model is integrated into the project, it 
will not be as effective. However, the major issues 
with mainstreaming lie in the time investment and 
barriers that need to be addressed by the cost model 
manager. Thinking through the possible barriers 
beforehand will enhance the effectiveness of the cost 
model as a tool. A final consideration is that the cost 
model is never done — it needs to constantly evolve to 
meet the needs of the project. 
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CASE STUDY 6: 

Washington State University 


A Cost/Benefit Analysis of Three Technology Development Strategies in Higher 
Education Using the Flashlight Economic Model 

Tom Henderson & Gary Brown 
Washington State University 


Abstract - Better information about the efficiency as well as the effectiveness of new 
technologies in higher education is essential , especially given the current rapidly changing 
environment in higher education. Washington State University has conducted a focused project 
using the Flashlight Economic Model to analyze the cost of three approaches to the 
development of learning technologies. This article has two purposes. First, to examine the 
effectiveness of the Flashlight Economic Model and the activity-based costing techniques it is 
built upon. Second, to present the results of the study that compared the costs of developing an 
online course, a stand-alone interactive module, and a template for Web-based computer 
conferencing. It appears the Flashlight Economic Model can be implemented without creating a 
"mudslide of data" and without special software. The WSU study also indicates that information 
from the model has the potential to be very useful. 

Revised June 1999 for the purposes of this Handbook 


Introduction 


Since ignoring new technologies in higher education 
is not an option, educational institutions across the 
country are now searching for effective strategies for 
investing in new technologies. No strategy promises 
to be cheap. The challenge is to find or develop 
approaches that are effective relative to cost. 

To determine the cost effectiveness of new 
technologies in education, the common approach has 
been to compare the costs of distance approaches to 
education with the costs of traditional education. The 
premise of the study reported here, however, holds 
that such a perspective, certainly for a rural, 
residential campus like Washington State University, 
is irrelevant. Online education is a competitive 
reality. We do not need to explore whether distance 
education is cheaper: we need to leam HOW we can 
accomplish it effectively -which necessarily includes 
consideration of costs. 

At WSU, like many institutions, we have tried 
several investment strategies for developing and 
integrating technology, usually simultaneously. 
Am ong them, a recent campus-wide Request for 
Proposal (RFP) process provided a good opportunity 
to assess three distinct approaches to technology 


investment. Two of the projects (or technology 
models) analyzed came from this RFP proposal while 
an instructor independently developed the third. 

1. The “modular approach," (InterFon™) targets 
one topic (phonemes) with the notion that the 
module might serve as a part of several courses. 
The development team included two content 
experts and one professional programmer from 
the Center for Teaching & Learning. 

2. The “Web-based conferencing template” 
(WebSeminar) allows faculty from any discipline 
to paste materials and assignments into the web- 
conferencing template and then facilitate the 
threaded discussions evolving from posted 
assignment. The development team included 
three professionals from The Center For 
Teaching & Learning and their partner unit, The 
Student Advising & Learning Center, as well as 
a number of student interns or hypemauts. 

3. The “single class model," (Teaching and 
Learning 445 or “T&L445) was developed by a 
single instructor of educational technology who 
already possessed both the resources and the 
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expertise to put a single, complete course online 
without the benefit of the granting process. 

The three models represented in this study, then, 
establish the context of the technology development 
strategies and shape the fundamental business (or 
cost-effectiveness) question: Which approach for 
developing online learning opportunities provided 
the most cost effective model for integrating 
technology into the university? As a critical 
qualification, we assumed, for the purpose of this 
study, that the quality and quantity of learning is held 
constant and subsequently qualify ’’effectiveness" as 
the least cost per volume of student use. This analysis 
did not address the larger aspect of benefits - 
learning - or the innumerable variables such a 
qualification entails. Clearly, a cost analysis will be 
significantly augmented when coupled with other 
approaches to assessment - including other Flashlight 
tools like the Current Student Inventory, the Faculty 
Inventory, and other planned assessment tools like 
the Alumni, and the Community surveys. 

The Problem: Costing Three Technology 
Development Strategies 

To answer the question about the efficiency of the 
three different development strategies, we also have 
to recognize that each of the technology approaches 
reflected a correspondingly distinct instructional 
implementation strategy that informs the analysis as 
well: 

1. The modular approach that targets a key course 
concept is designed for a single unit within a few 
Anthropology courses that focuses on teaching 
phonemes, the fundamental building blocks of 
language. In other words, the project would be 
used, by design , for only a few weeks during a 
single semester in any relevant course or courses. 

2. The web-based conferencing template was 
initially designed for and implemented in WSU's 
Freshman Seminars, which included 25 
introductory classes that were linked to general 
education courses. The linked courses, as it 
turned out, provided a good method for 
expanding the use of the web-based computer 
conferencing software because the companion 
courses, learning of the technology, often 
adopted the technology. 

3. The class conversion approach was used by an 
instructor in educational technology who asked 


students to read essays on technology in 
education and then e-mail responses to him from 
a designated window on the Web. The course 
design included graded responses, assignment 
sorting by dates, and other course specific 
features. The notable aspect of the class 
conversion approach in this study was that the 
design of this educational technology course for 
the Web integrated the course content with the 
technology being studied. In that sense, the 
course conversion reported here reflected a 
particularly comprehensive integration of a 
course for teaching online. 

The Method: The Flashlight Economic 
Model 

We adapted the Flashlight Economic Model to do the 
cost analysis study. The economic model has been 
developed as an evaluative tool as part of the 
Flashlight Program. The Flashlight Program was 
developed under the direction of Steve Ehrmann with 
a grant from FIPSE and Annenberg CPB to help 
educators assess new technologies intended to 
enhance and extend learning. Flashlight provides a 
suite of assessment tools like the Economic Model 
and the Current Student Inventory, as well as 
training, consulting, and other services. The 
Flashlight Program is now housed with The Teaching 
and Learning with Technology Group (TLT Group), 
an affiliate of the American Association for Higher 
Education. 

The Economic Model used in this study is a 
“...decision support tool which utilizes activity-based 
costing techniques to analyze the costs of campus 
outputs” (Thomas and Lovrinic, 1994). An activity- 
based costing (ABC) system models the usage of an 
organization's resources on the activities that use 
these resources and then links the cost of the 
activities to outputs or products (Lewis, 1995, p. 50). 
Figure 6-1 from the Flashlight Economic Model 
guidebook (Thomas and Lovrinic, 1994) illustrates 
these “links." 
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Figure 6-1: Organization-wide linkage of costs to activity units 
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The major difference between activity-based costing 
(ABC) and other accounting systems is that in an 
ABC model the resource costs are linked to activities 
rather than some other unit. For instance, teaching is 
an activity, and salary is the traditional cost measure 
associated with teaching. In education, however, just 
teaching may not accurately reflect the endeavor of 
developing an online course (see Table 6-1). ABC 
isolates the appropriate activities involved within a 
targeted study - such as integrating technology into 
teaching - and thereby provides organizations with a 
better understanding of the particular costs involved 
in that activity and a how to link those costs to the 
particular outputs being studied. By isolating the 
particular activities within a process, ABC also 
provides improved metrics and a better understanding 
of complex business processes and the associated 
costs. For example, results from this analysis 


revealed that as projects mature the percentage of 
resources spent on revisions and maintenance 
increases. This information has been useful for 
planning for the maintenance expenditures of future 
projects. 

The Traditional Accounting “view” vs. The 

Activity-Based Costing “view” 

The traditional view lists costs (or expenditures) by 
“general ledger account” while ABC categorizes 
costs by “activity units.” Allocating some costs can 
be relatively straightforward, e.g., the amount of time 
a person spends on each activity. Other allocations 
can be more difficult, e.g., an algorithm to correctly 
allocate capital costs. 


Table 6-1: Example of the traditional accounting “view” vs. The Activity-Based Costing “view” 


Department “X” traditional view 

Department “X” ABC view 

Salaries 

$30,000 

Instruction 

$20,000 

Benefits 

7,000 

Research 

15,000 

Supplies 

3,000 

Service 

15,000 

Equipment 

10,000 



Total 

$50,000 

Total 

$50,000 
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Seven Steps of the Flashlight Economic 
Model 

Seven basic steps are recommended to implement the 
economic model. However, “each institution is 
encouraged to modify these steps to fit their 
particular needs” (Lovrinic, Ehrmann & Banta, 
1999). 

1. Identify your business concerns and the specific 
questions you want answered. 

2. Identify your outputs. 

3. Identify the activities that go in to producing 
your outputs. 

4. Identify the academic and support units that 
produce your outputs. 

5. Identify the resources these units used to produce 
your outputs. 

6. Calculate costs for these activities using financial 
data about your resources: salaries and benefits, 
costs for supplies and equipment, building costs, 
and depreciation. 

7. Tally the costs of all activities to arrive at your 
overall costs. 

Step 1 - Identify business concerns and the specific 
questions you want answered 

The fundamental business question at WSU, framed 
by the three models studied, was to determine which 
approach for developing online learning opportunities 
provided the most cost effective model for providing 
technology-enhanced and online instruction. 

Step 2 - Identify outputs 

Outputs refer to the endpoints or constructs of the 
study. The outputs in this study surfaced the conflict 
between adding outputs to the model to increase the 
accuracy of the model, vs. limiting outputs to the 
model to constrain the study for purposes of 
practicality. For this analysis, the outputs were 
determined by the specific business question - the 
three development strategies: the module-based 
approach to technology development, the whole class 
conversion approach, and the web-based 
conferencing template approach. 

Defining the characteristics in each output that are 
similar and those that are different was essential in 
order to identify the activities involved in producing 
each output. 

The module approach : Required two faculty content 
experts and one programmer, hardware and software, 


targeted one learning concept for one class for a 
limited period of course time, with the possibility of 
further distribution into other courses later. 

The class conversion approach : Required only one 
faculty with expert technology skills, hardware and 
software, targeted an entire 15-week course. 

The web-based conferencing template approach : 
Required a design team, one lead programmer and 
several student assistants, hardware and software, and 
targeted an indefinite number of courses over an 
indefinite and flexible period of time. 

However, determining the level of detail in which to 
examine each of the outputs proved to be 
problematic, since each output studied had ’’more 
particulars than generalities,” making it difficult to 
draw generalizable comparisons. Nonetheless, the 
process of analyzing these outputs raised some 
valuable questions that provided insight into further 
analyses and strategies for developing and integrating 
technology into instruction. 

Step 3 - Identifying activities required to produce 
outputs 

The Flashlight Economic Model provides for 
flexibility and may be useful for a variety of studies, 
ranging from the comparison of newly adapted 
technologies, instructional training time, and others. 
Each study shapes the business question and the 
subsequent activities analyzed. 

There are several other considerations at this step. 
First, the design needs to address enough activities to 
distinguish between the outputs or projects. However, 
if too many activities are used, the distinction 
between them blurs. Another consideration is that 
different activities in activity-based costing may have 
different cost structures. For example, planning may 
be a different activity than implementing for several 
reasons; planning may not require actual teaching 
activities or the use of teaching assistants or 
classrooms but may require the purchase of software, 
the use of technical experts for programming, etc. 

Different models of “courseware engineering” and/or 
“instructional design” may be helpful when trying to 
determine activities for developing courseware. One 
of the first steps when conducting a study such as this 
required developing an "activity dictionary” listing 
the activities considered with definitions of each. 

The WSU study limited the number of activities 
because the analysis began after the new technologies 
and courses were either developed and in use or in 
the prototype stage. The estimated hours and other 
resources spent on each activity had to be determined 
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Table 6-2: Estimated hours spent by key developers for each activity 



Design & 
Prototype 

Implement 

Revise & 
Maintain 

Total 

Person A hours 

68 

346 

1,166 

1,580 

Person B hours 


346 


346 

RA, TA time 

- 

42 

123 

165 

TOTAL 

68 

734 

1,289 

2,091 


by post hoc interviews. The number of activities was 
therefore kept at a minimum to help the developers 
and content experts make clear distinctions and 
provide reasonable time estimates. It was decided to 
limit this economic model to three main activities: (1) 
design and prototype, (2) implement and (3) revise 
and maintain. The final model deviated slightly from 
strict adherence to activity-based costing. Some of 
the direct costs were allocated by activity determined 
by a series of extensive interviews and examination 
of applicable grants and other documents. However, 
some indirect expenditures were allocated on a “per- 
hour” basis, which did not reflect the fact that 
different activities probably used these resources at 
differing rates. 

Step 4 - Identify the units that participate in the 
activities 

Once activities are identified, the units involved in 
the production must be identified as well as the 
people in those units who performed the activities 
and tasks in order to produce the outputs. 

Table 6-2 gives a simple example of the activities in 
columns. The rows identify who performed the 
activities. The result is a two-dimensional matrix that 
shows how several individuals and/or offices take 
part in the same activities and tasks 


This example illustrates how different staff levels are 
involved. Person A could be a content expert while 
Person B could be a courseware developer. It is quite 
common to have a mix of courseware developers, 
content experts, and student or part-time labor 
involved in one project or even with one activity. At 
WSU, the units where most of the time accumulated 
for each course include: 

❖ Web Seminar included two major “units,” The 
Center for Teaching and Learning (CTL) and 
the Student Advising and Learning Center 
(SALC), 

❖ InterFon™ was produced by the CTL and the 
Department of Anthropology, 

♦> T&L445 was produce by the College of 
Education. 

Ste p 5 - Identify the resources the units consume in 
their activities 

The next step is to identify the resources these units 
and individuals use and apply prices to the quantities 
used. 

There are many possible types of expenditures. The 
attached “WSU model” categorized expenditures 
using two broad categories: direct and indirect 
expenditures. 
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Direct expenditures can be traced to specific 
expenditures for the projects (such as hardware, 
software, travel, consulting fees, etc.) and also consist 
of payroll, i.e., salary and wages from the estimated 
hours spent by person on each activity, including 
estimates for benefits. This analysis did not use 
depreciation to allocate the capital costs of each 
project over several periods; the costs of computers 
and software were directly “expensed” to each 
project to simplify the model. The useful life of 
computer hardware is becoming shorter and shorter 
and it was felt that amortizing these costs would not 
significantly impact the outcome of the analysis. 
However, the economic cost of using existing 
facilities (including computer hardware) was 

estimated by adding indirect expenditures for 
computing and maintenance. 

Indirect expenditures were estimated for such 
categories as goods & services, computing services 
(to estimate the use of the campus Ethernet 
connections, etc.), telephone, maintenance and 
utilities. Indirect expenditures for this project at WSU 
included estimates for goods & services, computing 


(mainly for the use of the Ethernet 
telecommunications system), telephone (telephone 
support was provided for the software development 
packages used, but WSU still paid for phone calls), 
and plant & operations maintenance. Once costs and 
cost drivers are identified, metrics had to be 
determined, i.e., by what standards of measure (or 
metrics) outputs are quantified. For this study three 
standards of measure were identified: total cost for 
the project, student weeks of use, and, the key metric, 
“total cost per student week of use”. 

Step 6 - Calculating activity costs 

Costs of activities will usually need to be estimated. 
Most university accounting systems do not use 
activity-based costing. In this way, ABC does not 
necessarily, (as one concerned faculty member once 
objected), "bury the audience in minutiae of detailed 
data.” The amount and detail of the data can be 
tailored to the kinds of decisions the data and the 
report targets. Issues such as this are summarized in 
Figure 6-2 (Cokins, 1996). 
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Figure 6-2: Common misconceptions about Activity-Based Costing 


Misconception 

Reality 

■ ABC requires a massive amount of data 
and detail, plus tremendous maintenance. 

■ The level of detail, accuracy and frequency of 
reporting depends on the kinds of decisions the 
data are to be used for. 

■ ABC data must be very accurate. 

■ It is better to be approximately correct than 
precisely inaccurate. 

■ It is tremendous work to collect a 
mudslide of data. 

■ Pareto’s 80/20 rule allows to significant amount of 
estimation from informed participants within a 
business process, combined with a few key 
interfaces to operational databases. 

■ ABC creates a separate set of financial 
books, which leads to confusion. 

■ Management can guide users as to which set of 
books to use. With time, organizations can 
integrate their general ledger with an activity 
accounting system. 

■ You cannot do ABC without special 
ABC software. 

■ The initial learning process can use spreadsheet 
software to translate general ledger data into 
activities. 


In this study, the first step was to interview key 
people in each project and ask them what their major 
activities were and to estimate the number of hours 
they spent on each activity. This, as we shall 
illustrate, was a key distinction in the analysis 
process. When possible, interviewees were asked to 
provide records to substantiate their estimates. Still, 
much of the attached data are based on the 
interviewees ’’best recollection.” A different 

interviewer may have approached the analysis 
differently, asked questions in a different way and 
may have found different (but hopefully similar) 
numbers. The major factor driving the costs for all of 
the projects analyzed was hours spent preparing, 
delivering and maintaining the courseware, both by 
content experts and technical developers. Once hours 
were determined, analysts could begin assigning 
payroll and benefits rates to compute payroll 
expenditures. Other direct expenditures were 
allocated to the projects (usually obtained from grant 
documents) and finally, indirect expenditures were 
estimated. 


Step 7 - Tally the costs of all activities to arrive at 
overall costs 

Several spreadsheets (or matrices) were used to 
separately analyze estimates of direct costs such as 
salaries & wages, payroll benefits, other direct 
expenditures such as hardware and software and to 
estimate indirect expenditures for selected activities. 
"Total expenditures" were derived by summing the 
detailed expenditures from the various spreadsheets. 

We envisioned the completed economic model as a 
pyramid constructed of stones, and each stone as a 
spreadsheet or matrix. The top stone, or highest-level 
matrix, contained data about the total estimated 
expenditures for all activities. The base of the 
pyramid included several stones, or matrices, each of 
which contained the estimated subtotals of overall 
expenditures - by each activity and related tasks. 

Figure 6-3 (from Lovrinic, Ehrmann & Banta, 1999) 
illustrates how we “worked up the pyramid” as 
subtotals were added from the lower matrices into the 
matrices at the next higher level. This resulted in a 
gradual tallying of estimated expenditures until total 
expenditures were attained for each activity. 
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Figure 6-3: Linkage of data from lower level “matrices” or spreadsheets to the top level 

summary report 



Collecting and Analyzing the Data 

Defining activities 

The Flashlight Cost Model can be adapted to use 
several measures of activities. Some activities, 
outlined by Lovrinic (1996) include: 

• developing concept/projections 

• preparing course content 

• preparing instructors 

• preparing other materials 


• marketing 

• recruiting and managing enrollment 

• instructing 

• evaluating and assessing 

Based on our experience at WSU, it may help to 
identify a list of activities and prepare an “activity 
dictionary” for use with the Flashlight Economic 
Model. Edward Forrest gives an example of a format 
for creating an activity dictionary that we reproduced 
in Table 6-3 (Forrest 1996). 


Table 6-3: Forrest’s template for an “activity dictionary” 


Activity 1 

Dictionary 

Activity Name 

Primary or 
Secondary? 

Work Driver 

Cost Object 

Performance 

Measure 

Comments 
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Determining and defining activities is perhaps the 
most important challenge, though identification of 
too many activities requires more timekeeping duties 
by all developers, at additional cost. 

Measuring the volume of use 

In his book on activity-based costing, Lewis (1996) 
specifically notes the implications of activity-based 
costing (ABC) for education. He suggests using 
credit hours as a measure of volume. However, the 
projects examined in this study, like many 
educational innovations, are often not used for an 
entire semester. Like the Carnegie unit in general, 
new technologies in education are continuously 
challenging the credit hour as the appropriate 
measure for conceptualizing education or educational 
activities. 

So far, the whole course conversion strategy used for 
Teaching & Learning (T&L 445) has used the 
innovative online approach for the entire semester. 
However, InterFon, the technology module, was 
designed to be used during 3 to 5 -week intervals in 
one of three courses. WebSeminar has been used by 
many courses for the entire term, but several courses 
used it for four-week sections. Therefore, rather than 
dividing activities by credit hours, we tracked volume 
by the number of students who used the new teaching 
technologies multiplied by the number of weeks 
students used the innovation. We called this metric 
"student weeks." 

"Student weeks," like any metric that attempts to 
contextualize the use of an innovation, has 
limitations. "Student weeks" at any given point 
within a study will not reflect the potential use of a 
product. In this study, the use of WebSeminar (now 
Speakeasy) has expanded dramatically. The use of 
InterFon™ has expanded beyond our predictions into 
new applications. Enrollment and therefore the use of 
T&L 445 has also expanded, though obviously not 
into other courses. 

Student weeks as a measure also fails to reflect the 
distinction between a two and three credit course, 
which may have various impact on measures of use. 


Gathering data 

An additional hurdle we encountered was that WSU 
presently has no central repository for the kinds of 
data needed for this type of economic analysis, 
especially the estimates of cost and allocation of 
"indirect" expenditures. It was therefore necessary to 
gather data from a variety of sources on campus, 
including Institutional Research, the Finance Office, 
and other sources. We had to track down cost data by 
speaking with administrators responsible for capital 
budgeting and operating budgeting and had to access 
grants and RFP documents. Furthermore, it proved to 
be somewhat difficult explaining to the appropriate 
administrators the types of data needed. For example, 
we went to Person A, who sent us to person B, who 
sent us to Person C, who said we really should see 
Person A. Certainly some of the difficulty stemmed 
from our inability to communicate across various 
administrative discourse communities. Some of the 
difficulty reflects the fact that activity-based costing 
is relatively new (invented during the last decade). 
ABC originated in industry and has been adapted by 
"service organizations" even more recently, the 
Flashlight Economic Model is even newer. As 
educators become experienced collecting cost data 
they should become familiar enough with the 
accounting system of their institutions to find most 
necessary data. 
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Results 

Figure 6-4 illustrates the total costs of the three 
projects. Please note: InterFon™ was partially funded 
by a grant from Boeing and also WebSeminar (now 
the Speakeasy Studio) was largely funded by grants 
from Microsoft and Boeing. The total costs estimated 
below do not “net out” any of the grant benefits. The 
cost of the web-based conferencing template, 
WebSeminar, was substantially more expensive to 


develop than the other two approaches, costing 
roughly $180,000 to the point of instructional 
implementation and reliable use. The module project, 
InterFon™, was the next most expensive 
development strategy, costing $40,000 to develop to 
the point of reliable implementation and classroom 
use. Finally, the cost of converting an entire class was 
the least expensive strategy. 


Figure 6-4: Estimated total cost of the three projects analyzed through 12/97 
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However, the total cost of the projects is a misleading way to compare the cost efficiency of the three development 
strategies. A more accurate measure may be the total cost per unit of volume. Figure 6-5 illustrates the estimated 
amount of student weeks of use through the time of the initial study. 

Figure 6-5: Estimated student weeks of use through 12/97 


Estimated Number of "Student Weeks" of Use through 12/97 



The estimated student weeks of use for the web- 
based conferencing template, WebSeminar, was 
substantially higher than the other two approaches, 
with an estimated 24,000 student weeks of use. The 
module project, InterFon, had the least amount of 
use, at a user level of 28 student weeks (InterFon had 
just been prototyped for one week in a class of 28 
students). There were 95 students in the T & L 
course who used the online conversion for the entire 
semester giving a use rate of 1,425 student weeks of 
use. 

Finally, to put the costs to develop each module and 
the amount of student use into perspective, we 
divided the cost to develop each course by the 
number of student weeks of use. Figure 6-6 illustrates 
the cost per student week of use. 


The cost per student week of use for the web-based 
conferencing template, WebSeminar, was about 
$8.00 per student week, whereas the cost of the 
modular approach was $1,400 per student week. The 
course conversion also cost roughly $8 per student 
week of use. It had a lower initial cost than the 
WebSeminar but fewer students had used the online 
course than those who used the WebSeminar. 
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Figure 6-6: Estimated average cost per student week of use through 12/97 
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Conclusions 

The salient finding of this study is: even though the 
web-based conferencing template costs substantially 
more to produce than the module approach, its cost 
per student week is comparable to the course 
conversion approach used for Teaching and Learning 
445 and much less expensive when compared to the 
InterFon module. 

This finding is not surprising, though the magnitude 
of the difference perhaps is. Modules designed for 
just a few weeks of use in specific classes will not be 
used by as many students as software that can be 
used for the whole term and in many courses. 
Further, as modules get increasingly sophisticated, 
such as InterFon, they become increasingly expensive 
to produce. 


The Flashlight Model did a good job of pinpointing 
another important issue, that is, as a new technology 
continues to be used, the proportion of total expenses 
used to revise and maintain it increases. WebSeminar 
had been used for several semesters and had (and 
continues) to go through several revisions. T&L445 
was in its first semester of use and InterFon was just 
being prototyped. Figure 6-7 shows that the ratio of 
expenses for the “Revise and Maintain” activity 
which appears to be increasing proportionally with 
the maturity of the product. 
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Figure 6-7: Cost by major activity as a percentage of total estimated costs to-date 

through 12/97 



This final observation may have major implications 
for institutions developing new technologies. The 
cost of the new technologies will continue to rise as 
they are used. Revisions and maintenance costs seem 
inevitable and may become a major cost of the 
product as it matures. 

The first finding reporting the general costs and even 
the costs per student week could have been reached 
by analyzing total costs using traditional “general 
ledger” accounts. However, the additional and 
imperative finding that suggests upgrades and 
maintenance costs need also be considered in the 
development cycle, though they appear to be obvious 
when the costs are categorized by activity using the 
Flashlight Economic Model, would probably not be 
so obvious if we had selected to use traditional cost 
estimates based strictly on general ledger accounts. 

Additional Considerations 

The modular approach to technology development 
(InterFon), which targeted one part of one course or a 
single concept, did, it turns out, have a spin-off 
application. The spin-off was entirely unanticipated. 
However, the unanticipated utility - the preservation 
of the phonetic components of dying Native 
American languages - underscores one aspect of the 
problematic nature of a cost/benefit analysis. First, in 
the case of InterFon, the ultimate valuation of the 


benefits of this program, or any program of this 
nature was dependent upon great fortune - or great 
foresight, which WSU, for this project at least, cannot 
claim. 

Second, though the language preservation application 
of the program in this study could be counted as an 
intangible benefit, there may also be a rather 
substantial cost. Any software program requires 
upgrading and maintenance if it is to persevere, and 
that cost will persist throughout the life of the 
program both in real and in opportunity costs. 
Whoever maintains the program is not doing 
something else. So the program must be maintained 
or responsibility transferred to a reliable, perhaps 
commercial, organization. 

There may be a variety of ways to do this, including 
burning the program and preserved language into a 
permanent storage medium such as a CD-ROM. But 
there are still a number of issues necessary to assure a 
reasonable life span of the product, and such 
activities generally do not fit snuggly, at this point, 
into educational institutions daily business. 
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The entire class model (Teaching and Learning 445) 
was developed by an instructor of educational 
technology who already possessed both the resources 
and the expertise to put a single, complete course 
online. One significant advantage to this approach is 
that the process of developing the course also keeps 
the professor involved and current in his profession. 
Likewise, maintenance and upgrades, already 
significant in this project, very easily qualify as 
professional development that may benefit students 
as well as the instructor. On the other hand, that the 
instructor in this study was an instructor of 
educational technology also limits the generalization 
of many of the benefits noted here, particularly those 
that count as professional time-on-task. Our RFP 
process with faculty who targeted full classes and 
who were funded were generally not so successful, 
many of whom have yet, almost three years later, 
completed their efforts. Funding faculty to develop 
their own courses depends heavily on the technology 
skills of the instructor involved. There is no evidence 
in this study that suggests that the approach would be 
successful with faculty in disciplines other than 
educational technology. 

A multidisciplinary learning space (WebSeminar, 
now upgraded and called the “Speakeasy Studio” 
after its latest revisions) allows faculty from any 
discipline to paste materials and assignments into the 
“learning space” on the WWW and then to facilitate 
the text-based interaction around that material. 
However, the broader the base of use, the less the 
customizability of the application remains an 
advantage over commercial developers, who are 
making and serving learning spaces that are 
increasingly available and that eliminate the 
university's responsibility for providing upgrades and 
maintenance. 

In the final analysis, the three kinds of development 
outlined in this study may have more particulars than 
generalities, which qualifies the categorization of 
module, class and web-based conferencing template 
in particular ways. Each approach raises different 
questions. Each presents particular opportunities and 
catwalks particular pitfalls. 

Limitations, Caveats, and Further Study 

The absence of benchmarks 

This analysis did not conduct a ’’make vs. buy” 
analysis. What is the lifecycle of courseware or new 
technologies in education? Will the cost of 
maintaining the systems be overwhelming? How 
many revisions are practical to meet requirements of 
new hardware, new versions of operating systems, 
communications systems, etc? Some of these 


questions suggest the ongoing nature of cost analysis. 
Depending upon the purpose of the study, the 
absence of benchmarks may be worth considering. 
Many problems with programming or adapting a 
particular ’’courseware” system are addressed in 
Ehrmann (1995). 

Strategic retum-on-investment indexing 

The analyses presented here may be improved if they 
had included a Retum-on-Investment analysis (or 
index) and/or comparing the Net Present Values of 
the cost of each technology. Such calculations would 
factor in the time- value of money and should be kept 
in mind for studies of investments in new 
technologies. 

Context assessment 

This analysis did not address the larger aspect of 
benefits - learning. Clearly, a cost analysis will be 
significantly augmented when coupled with other 
assessments - including other Flashlight toolkits like 
the Current Student Inventory, the Faculty Inventory, 
the planned Alumni and Community surveys. 

For example, the T&L 445 course has just adapted 
new teaching technologies. A survey of regional 
principals who hire graduates from the WSU program 
will provide an additional depth to our perceptions of 
benefits that were not possible with the single model 
of assessment presented here. We are planning on the 
opportunity to enrich our cost assessments by 
integrating these additional tools. 

The perceptions of benefits 

We also discovered that many faculty and staff are 
incensed at putting a dollar value on the benefits of 
education. When this analysis was presented at a 
faculty forum in the spring of 1998, there were 
faculty who found the whole majesty of ascribing 
benefits to the learning process - tangible or 
intangible -disconcerting and even upsetting. One 
particularly hostile faculty member disrupted the 
presentation. His concerns, we suspect, reflect a 
general perception held by faculty that the business 
of teaching and learning is significantly different 
from other businesses to merit the same measures of 
scrutiny. 

When we presented the cost/benefit model to the 
University's chief budget offer, he confirmed, from 
his experience, similar attitudes. Faculty will damn 
you if you do a cost/benefit analysis, and the public 
and funding constituencies will damn you if you do 
not. 
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Certainly, it is necessary to be sensitive to faculty 
concerns. And certainly any attempt to quantify 
benefits is, ultimately, reductive. On the other hand, 
the process of such an analysis helped us revisit our 
values and contextualize them into the world and into 
the language of legislatures and taxpayers. It is 
becoming increasingly clear to those of us in higher 
education that we need to continually communicate 
with our constituencies, and developing a common 
language is an essential aspect of 
any communication. 

Several studies have analyzed the benefits of a 
college education as a whole, both to individuals and 
to society. Examples of personal benefits include 
improved individual knowledge, productivity and job 
market search efficiency; examples of benefits to 
society measure effects such as intra-family 
productivity, family health, social cohesion, and other 
benefits (Haveman and Wolfe, 1984). Another study 
used “data envelopment analysis” to attempt to 
measure the added value of top MBA programs in the 
U.S. (Haksever & Muragishi, 1998). Other studies 
have measured the economic impact of a particular 
university on a state’s economy (Clark, Feng & 
Stromsdorfer, 1998). These studies may use different 
approaches, but they all measure the impact of a 
degree or program or of a total university, not a 
single course. 

However, analyzing and trying to quantify the 
benefits of technology derived from individual 
courses is difficult and subjective. 

Finally, if we fail to provide our analysis of the costs 
and benefits of our educational efforts, we forfeit the 
challenge to those who will, which, like assessment 
in general, often turns out to have more unpleasant 
consequences than the wrath of our colleagues. 
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CASE STUDY 7: 


The University of Hong Kong 
Chasing The Great White Whale Of Open Access Costs 

Craig Blurton, Anita Lee, and Winston Ng 

The University of Hong Kong 


Abstract: In 1998, the University of Hong Kong instituted a campus-wide notebook computer 
programme. Among other benefits, the programme furnishes an alternative method of providing 
students with access to a computer and the network. At the moment, HKU continues to offer 
computer and network access to students in computer laboratories as well as supporting the 
notebook computer programme. But, in light of increasing demand and decreasing resources, 
decisions must eventually be taken about how best, and most cost-effectively, to provide open 
access. Using the Flashlight methodology, in 2000, the IT & Teaching Group undertook a cost- 
analysis study to answer three questions concerning the provision of open access: (1) How much 
did each of the two methods of providing open access for students cost? (2) Who paid for which 
costs, e.g., central budget, faculty or departmental budgets, students, other funding sources, or 
vendors? (3) Which method of providing access - laboratories or the notebook computer 
programme - was most cost-effective from the University's perspective? 


LOOMINGS 

What the White Whale was to Ahab, has been 
remains unsaid. 

Herman Melville 


Call us naive. Like Ahab in Melville’s classic 18 th 
century novel Moby Dick , who went in search of a 
leviathan, “a Sperm Whale of uncommon magnitude 
and malignity, which whale, after doing great 
mischief to his assailants, had completely escaped 
them.” (pp. 194-195), we boldly and fearlessly 
lowered away in search of the Great White Whale of 
information technology (IT) costs at the University of 
Hong Kong (HKU). Although “chancing only to hear 
of [them] distantly and vaguely,” we specifically 
went in search of the costs of providing students with 
access to a computer and the network. 

ALL ASTIR 

A day or two passed, and there was great 
activity aboard the Pequod. Not only were 
the old sails being mended, but new sails 
were coming on board, and bolts of 
canvas, and coils of rigging; in short, 


hinted; what, at times, he was to me, as yet 

Moby-Dick, or The Whale 1 

everything betokened that the ship's 
preparations were hurrying to a close. 

(p- 104) 

Our work commenced in 1998 when the University 
of Hong Kong (HKU), after a long period of 
planning, entered into a three-year “Partnership” with 
IBM China/Hong Kong Limited, one component of 
which was a notebook computer programme. Under 
the terms of the Partnership agreement, IBM agreed 
to supply all incoming freshmen with a ThinkPad 
notebook computer at a significant discount. The 
University agreed to provide a subsidy, making the 
final cost to each student for the complete notebook 
bundle approximately 20-25% of the equivalent retail 
value. To ensure network access, the HKU Computer 
Centre created “ACEnef ’, a “plug-and-play” network 
that provides more than 10,000 access points across 
campus in the Library, dormitory rooms, classrooms, 
laboratories, hallways, and other places. 
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Altogether, after three years of the Partnership, 6,799 
undergraduate students (about 82%) have ThinkPad 
notebook computers and are able to access a 
computer and networked resources from any place 
on- or off-campus at any time. The University, 
students, and IBM have invested significant resources 
in the programme. 

When being planned by an ad hoc committee of 
academics and administrators, there were three major 
reasons put forward as to why the University should 
initiate a notebook computer programme: 

1. Anywhere, anytime access to a computer would 
enhance the quality of education available to 
HKU students. 

2. Ownership of a personal computer would 
improve students’ information technology skills 
and knowledge. 

3. The University would realize cost savings from 
instituting such a programme. 

In 1999, as part of our responsibility to provide 
University decision-makers with information 
concerning information technology (IT) use on 
campus, we (the IT & Teaching Group within the 
Centre for the Advancement of University Teaching 
[CAUT]) decided to look closely at the third reason, 
and study the question of costs. 11 Crowding on all 
sail, we pressed ahead with the fair, fresh wind of 
enthusiasm to urge us along. As envisaged, our study 
would compare costs between two methods currently 
used on campus - the notebook computer programme 
and open access computer laboratories - and would 
be completed within a month or two. 

THE ADVOCATE 

“I was thinking of shipping. ” 

“Thou wast, wast thou? I see thou art no 
Nantucketer - ever been in a stove boat? ” 

“No, Sir, I never have. ” 

“Dost know nothing about whaling, I dare 
say -eh?” 

“Nothing, Sir; but I have no doubt I shall 
soon learn. ” (p. 79) 

Thus we began our study. We were new to 
conducting a cost-analysis study, although we are all 
experienced educational researchers. If we had been 
asked at that time what we knew about conducting a 
cost-analysis study, we might have replied, like 
Ishmael when signing aboard the Pequod, “nothing,” 
but we had no doubt we could learn. 

In retrospect, perhaps our decision to conduct the 
study was naively made, without sufficient 


understanding of the difficulty of the task. But we set 
off on our voyage anyway, with the one only and all- 
engrossing object of hunting the White Whale of the 
costs related to providing students with access to 
computers and the network from on- and off-campus. 

Like Ahab, the path to our fixed purpose was laid 
with iron rails. Over unsounded gorges, through the 
rifled hearts of mountains, under torrents’ beds, 
unerringly we did rush. A study that we thought 
would take a few months turned into almost a year’s 
work. The data that we needed and thought would be 
simply available to us from one or two central 
support unit databases turned out to be buried deep in 
hundreds of filing cabinets across all three of HKU’s 
campuses. 

THE SHIP 

She [is] a ship of the old school ... with an 
old-fashioned, claw-footed look about her. 

Long seasoned and weather-stained ... 
her ancient decks were worn and 
wrinkled. ... But to all of these her old 
antiquities, were added new and 
marvelous features ... (p. 77). 

To understand this study, and the environment in 
which it was conducted, it is necessary to know 
something about the University of Hong Kong 
(HKU). Sir Frederick Lugard, Governor of the British 
Colony, laid the foundation stone for the oldest 
tertiary institution in Hong Kong, on March 16, 1910. 
Two years later the University officially opened with 
two founding faculties, the Faculty of Engineering 
and the Faculty of Medicine. In December 1916, the 
first class of 28 students graduated. 

Since then, HKU has grown from two faculties to ten, 
with the addition of faculties of Architecture, Art, 
Business and Economics, Dentistry, Education, Law, 
Social Sciences, and Science. Today there are 14,000 
students (about 9,000 undergraduate, 5,000 graduate 
and 650 international) and more than 1,000 full-time 
teaching staff. 

Since its humble beginnings, HKU has become one 
of the pre-eminent international universities in Asia. 
In 2000, AsiaWeek Magazine ranked HKU third 
overall among all multi-disciplinary universities in 
Asia. The two institutions outranking HKU, Kyoto 
University and Tohoku University, teach primarily in 
Japanese. In research, the University ranks 
consistently among the top few research universities 
in Asia and compares well to the best North 
American institutions. 

Since 1997, the University has begun a major 
restructuring of its curriculum to incorporate 
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problem-based learning, cross-curricula activity, 
cross-cultural sensitivity, and to ensure that all of its 
students achieve information technology (IT) 
literacy. These efforts have paid major dividends in 
improved student learning, the renewal of an 
atmosphere of excitement and innovation, and 
increasing stature as a leader among universities 
worldwide. Thus, HKU may be accurately described 
as traditional, conservative, very successful, and 
evolving to meet new challenges. 

MOBY DICK 

For as the secrets of the currents in the 
seas have never yet been divulged, even to 
, the most erudite research; so the hidden 
ways of the Sperm Whale when beneath 
the surface remain, in great part, 
unaccountable to his pursuers; and from 
time to time have originated the most 
curious and contradictory speculations 
regarding them ... (pp. 197-198). 

IT has become an increasing and on-going expense in 
every educational institution’s budget. Some 
estimates of the amount spent from University 
operating budgets on IT range from one percent to 25 
percent, with an average of about five percent 
(NASULGC, 1999, pp. 10-11). But, if one looks 
closely enough, these costs are rarely well understood 
or documented. 

At HKU, as elsewhere, there has been growing 
pressure to provide students with more and more 
access to computers and the network due to the 
increasing use of such resources in educational 
programmes. As the need to provide student access 
has increased, HKU administrators have been faced 
with growing, and in some cases unforeseen, costs. 
When we began our study, those costs seemed 
hidden, unaccounted, curious, and contradictory. 

Our Moby Dick, the fundamental resource issue we 
addressed, was how best - and cost-effectively - the 
University could provide students with access to 
computer and networked resources. Prior to 1998, 
student access to computers and the campus network 
(and hence the Internet) for general-purpose use at 
HKU was provided only by means of “open access” 
computer laboratories, some centrally maintained by 
the Computer Center; others set-up by individual 
faculties, departments, or other academic units. The 
workstations in open access labs are pre-loaded with 
a variety of general productivity or network software 
including word processors, spreadsheets, web 
browsers, email applications, and so forth. 


What distinguishes open access labs from other 
computer labs on campus is that they are not typically 
used for instruction, i.e., to host a computing class 
with an instructor, and they are not equipped or 
designed for discipline-specific use, e.g., a GIS 
computer lab for Geography or a CAD/CAM lab for 
Architecture students. 111 

Since 1998 the HKU/IBM Partnership has provided 
incoming students with an alternative means to 
access a computer and the network. Students may 
purchase a notebook computer and plug into ACEnet, 
a “plug and play” network created on campus for 
ubiquitous network access. In addition, the capacity 
of the Computer Centre’s modem pool that provides 
off-campus access to students and staff was greatly 
enhanced. 

At the moment, HKU continues to offer computer 
and network access to students in computer 
laboratories as well as supporting the notebook 
computer programme. But, in light of increasing 
demand and decreasing resources, decisions must 
eventually be taken about continuing the notebook 
programme, maintaining open access laboratories, or 
developing some approach inclusive of both the 
notebook computers and laboratories. And, in either 
case - open access labs and the notebook computer 
programme - the question of who bears which 
specific costs - the University, faculties and 
departments, and individual students - is an important 
factor that HKU administrators need to consider in 
making resource decisions. 

Cost Analysis Model 

The cost analysis model we used for the study was 
based on the seven-step “activity-based costing 
(ABC)” model in the Flashlight Cost Analysis 
Handbook, Version 1.0 (Ehrmann & Milam, 1999) 
published by The Teaching, Learning, and 
Technology Group (TLT Group): 

• Identify specific questions. 

• Identify the outputs. 

• Identify the activities that go into producing 
the outputs. 

• Identify the academic and support units that 
produce the outputs. 

• Identify the resources these units use to produce 
the outputs. 

• Calculate costs for these activities. 

• Tally the costs of all activities to arrive at 
overall costs. 
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The Handbook provided us with scaffolding upon 
which we could construct our study for which we 
were very grateful. We read it carefully, scrutinized 
the case studies in the appendices, and embarked on 
our voyage. 

Specific Questions 

Our cost analysis was intended to answer three 
questions: 

1. How much did each of the two methods of 
providing open access for students cost? ,v 

2. Who paid for which costs, e.g., central budget, 
faculty or departmental budgets, students, other 
funding sources, or vendors? 

3. Which method of providing access - 

laboratories or the notebook computer 

programme - was most cost-effective from the 
University's perspective? 

Output 

Determining the fundamental output of interest was 
more difficult than first realized, and - in the end - it 
was one of the more controversial decisions made. As 
the Flashlight Manual affably notes, “there are many 
ways to express outputs” (Ehrmann & 

Milam, 1999, p. 7). 

In our case, the fundamental resource issue we were 
addressing was how best the University could invest 
its limited resources to provide students with access 
to computers and the network. But which students 
should we include and what specific metric or 
indicator, or “output”, should we use? Should we 
attempt to include open access costs for all students, 
undergraduate and postgraduate, or limit the study to 
undergraduate students only? In the end, we had to 
limit the study to undergraduate students and ignore 
costs that might have been attributed to graduate 
students for two major reasons. 

First, the notebook computer programme was 
primarily intended to serve the undergraduate student 
population. Although postgraduate students and staff 
are offered the opportunity to purchase a notebook 
computer, they do so without a University subsidy 
and at a less favorable discount from the vendor. 
Therefore, we knew that few postgraduate students 
had participated in the notebook programme and we 
had no way of knowing whether or not they already 
owned a notebook computer purchased from a 
different source. We could not determine what costs, 
if any, were associated with graduate student use of 
notebook computers and therefore, for the purposes 
of the study, we assumed there were none. 


Second, we had no reliable way of ascertaining how 
postgraduate students across campus in the various 
disciplines were being provided with access to a 
computer and the network in other ways. We did 
know from anecdotal evidence that the provision of 
access varies considerably from academic unit to 
academic unit and from discipline to discipline. In 
some departments, individual postgraduate students 
are provided with a workstation and network access 
from grant monies. In other departments, two or more 
postgraduate students in an office suite are provided 
with access to a single computer. Some departments 
do not provide any computer or network access to 
postgraduate students, in which case they may 
purchase and use their own notebooks or 
workstations on campus, or simply do without. And, 
of course, postgraduate students do have access to 
workstations situated in the campuses’ open access 
laboratories. 

The latter factor was ignored. That is, when 
determining open access laboratory costs per student, 
we did so without consideration of postgraduate 
student use. We had no way of tracking how many 
postgraduate students make use of computers in open 
access laboratories nor how often they use them. We 
also had no evidence that if postgraduate student use 
of computers in laboratories were to be disallowed, 
that the number of such laboratories, or their 
associated costs, would decrease. In fact, the opposite 
may be true as more departments are now opening 
such labs. But excluding the costs associated with 
providing laboratory access to postgraduates, who 
make up 35% of the total student population, has 
been a point of controversy. 

The second point of controversy about our choice of 
“output” has been our decision to use costs per “hour 
of accessibility” as a metric. We did not consider 
students' actual hourly use of either open access 
laboratory workstations or notebook computers, but 
rather compared the hours individual computers were 
available for use. In the case of open access 
laboratories, this meant calculating the number of 
computers times the number of hours each laboratory 
was open to students. In the case of notebook 
computers, since students own them and could in 
theory have them with them at all times, this meant 
calculating the number of students times the number 
of days in the academic year times 24 hours a day. 
This has been seen as an unfair comparison, since 
students do not usually use their notebook computers 
24 hours a day. 

We decided to focus on access rather than use 
because we could measure one but not the other. We 
could measure the hours of access available in 
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computer laboratories but could not obtain reliable 
data about student use of individual computers in 
these labs. Such data do not exist and are not 
collected/ And we also reasoned that anywhere, 
anytime access had additional value in the 
educational process to notebook using students that 
could not be included in the study quantitatively, but 
which could help justify our approach. 

Second, we focused on access rather than use because 
the University can determine whether or not to 
provide lab access and at what level, but it cannot 
dictate nor too accurately project student demand. 
Demand varies greatly from discipline to discipline, 
week to week, day to day, and student to student/ 1 
Engineering students probably need more access than 
students in the Arts. And, any attempt to measure 
student use of workstations in open access 
laboratories would be unable to factor in whether 
more use would have been possible, indeed desirable 
from the students’ point of view, had more 
workstations been available at any given time. 

In conclusion, we picked the cost of providing access 
to a computer and the network per hour as the 
“output” for the study. However, we also considered 
costs vu from a variety of other perspectives as you 
will see. 

THE FIRST LOWERING 

“There she blows! There! There! There! 

She blows! She blows! ’’ 

“Whereaway? “ 

Instantly all was commotion. 

“Lower away then; d’ye hear?’’ shouting 
across the deck. “Lower away there, I 
say. ” ... the sheaves whirled round in the 
blocks; with a wallow, the three boats 
dropped into the sea ...” 
(pp.234; 236-237) 

We believed our data collection efforts would be 
limited to a simple phone call or two. How little we 
understood the situation! The data necessary to 
complete the study for identifying the actual cost of 
PC laboratories were not readily available from a 
central source, for reasons of complexities of funding 
sources and differences in time and price for the PCs 
and other fitting-out works. 

The Finance Office (FO) handles nearly 100,000 
purchase orders every year for items paid for from 
many sources of funding including a central IT 
budget for the Computer Centre (CC), individual 
one-line budgets for academic units, donations, 
grants, and so forth. And large individual purchase 


orders are often used to buy multiple items, some 
relevant to our study and some not. 

In order to provide us with the necessary information 
concerning IT purchases and expenses at the 
academic unit level, someone in the FO would need 
to examine thousands of purchase orders, scrutinizing 
each and individually itemizing those costs of interest 
to us. Doing so would simply be too time consuming 
and costly. Plus, whatever data they did have 
concerning IT expenditures by faculties, departments, 
and other academic units would not be adjusted to 
reflect the depreciation of equipment purchased in 
previous years. 

The issue is further complicated at HKU by two other 
factors. First, because academic unit funding is by 
one-line budget, there are fewer restrictions on how 
the money may be spent, i.e., one department may 
choose to spend a higher percentage of the total 
budget on IT-related costs while another spends less. 
Second, although the University does negotiate 
pricing with major IT vendors centrally through the 
CC, individual departments may choose to purchase 
IT directly from vendors, negotiating pricing and 
maintenance contracts individually for specialty 
items of equipment or software. This system provides 
a higher degree of autonomy to academic units, but it 
also became necessary to collect data from each unit. 

LESS ERRONEOUS PICTURES 
OF WHALES 

And all the while the thick-lipped 
leviathan is rushing through the deep, 
leaving tons of tumultuous white curds in 
his wake, and causing the slight boat to 
rock in the swells like a skiff caught nigh 
the paddle-wheels of an ocean steamer. 

Thus, the foreground is all raging 
commotion; but behind, in admirable 
artistic contrast, is the glassy level of a 
sea becalmed ... (p. 291) 

In February 2000, we began meeting with 
representatives from the FO and the Computer Centre 
(CC) to solicit assistance and advice. Staff from these 
organizations provided invaluable assistance 
throughout the study. The FO explained the 
accounting procedures for IT purchased centrally, 
identified how costs were calculated and depreciated, 
provided extensive data concerning campus-wide 
costs (i.e., paid for from the central budget), and 
volunteered to serve as liaison with the campus 
Estates Office which is responsible for building, 
renovating, and maintaining the HKU physical plant, 
and hence bore some of the expenses for both open 


131 



access programmes. They also strongly advised us to 
use a "standard cost” approach for the cost 
comparisons, rather than the ABC approach. The CC 
staff provided descriptions of all centrally managed 
open-access laboratories as well as the costs related 
to equipping, maintaining, and operating them. 
Furthermore, they provided data about the costs of 
creating ACEnet. 

Having established precisely what information would 
be available centrally concerning open access 
laboratories and the notebook computer programme, 
we met informally with a few friendly heads of 
Departments to explain what we were trying to 
accomplish, and solicit their feedback about what 
kinds of information about IT expenses could be 
gathered directly from academic units. 

These discussions led to two important decisions. 
First, we would need to limit the amount of 
information requested because in most cases 
overworked Departmental secretaries would be 
responsible for collecting the data for us and would 
be much less likely to cooperate with the study if 
doing so were to be too time consuming. And second, 
we would have to accept estimates for some costs 
because departmental records were often incomplete 
or, as in the case of the FO, costs which we were 
interested in were buried in bulk purchase orders and 
for all intents and purposes unidentifiable. 

LEG AND ARM 

“Hast seen the White Whale? ” (p. 476) 

We decided to use two methods to solicit information 
from individual academic units, email requests for 
information and a self-administered Excel 

spreadsheet also dispatched by email. Because we 
were, in a sense, comparing apples and oranges when 
comparing the costs related to open access 
laboratories and the notebook computer programme, 
we treated each as a separate study. For each we 
followed the Flashlight ABC approach identifying 
specific sub-questions, outputs, activities that 
produced the outputs, academic and support units that 


helped produce the outputs, resources used by these 
units, then calculated the costs for the activities, then 
finally tallying all of the costs for both approaches. In 
the end, we compared the costs per hour of access 
between the two methods. 

THE CHASE — FIRST DAY 

“There she blows ! — There she blows ! A 

hump like a snow-hill! It is Moby Dick!” 

(p. 595) 

Study of Open Access Laboratories 

In the case of open access laboratories, the sub- 
questions were: 

• How much does it cost to provide one hour of 
access to a computer and the network for 
undergraduate students through open access 
computer labs/workstations? 

• Who pays for what in providing the computer 
and network access in open access laboratories, 
i.e., what costs are funded from the University’s 
central budget, faculty or departmental budgets, 
and other funding sources respectively? 

The output was one hour of access to a networked 
computer in an open access lab. The activities 
required to produce this output included the setting 
up, operation, and maintenance of the computer labs 
and workstations. As for the organizational units, at 
HKU there could be as many as 68 units directly or 
indirectly involved in providing access by means of 
open access laboratories including the Computer 
Centre, Estates Office and Library at the campus- 
wide level, nine faculties and one school at the 
faculty levd VI11 , and 55 academic departments/units 
(including the English Centre and Language Centre) 
at the departmental level. 

Regarding the inputs or resources consumed in 
carrying out the activities, with the help of the 
Finance Office and the Computer Centre costs were 
identified in two general categories: setup costs and 
recurrent cost. 
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The specific input or cost items under each category were as follows: 


Setup Costs 

Recurrent Costs 

. Hardware & software 

. Hardware maintenance agreements 

. Network connections 

. Hardware & software upgrades 

. Fan coil unit system for air- 

. Staff support - maintenance, problem fixing, 

conditioning 

general administration 

. Lighting 

. Space (rental rate) 

. Other electric appliances 

• General maintenance - lamp replacement, fan coil 

. Builders (for renovations) 

unit maintenance, minor repairs to furniture, minor 

• Furniture 

repairs to door locks, ironmongery, false ceiling 
tiles, general cleaning 

. Management charge - apportioned operation & 
maintenance costs of lifts and main chiller water 
plant of the building 

. Utilities - lighting, power for PCs and 
air-conditioning 


For the 65 academic units, a self-administered Excel 
spreadsheet was sent to the unit heads via email to 
collect relevant data. Heads were requested to fill in 
the spreadsheet with the following information 1 * and 
return it electronically: 

• The number of workstations in individual labs 

• The total weekly access hours of individual labs 

• The estimated total setup cost 

• The estimated total annual recurrent cost 

• The estimated proportion (in percentage) of these 
costs being funded from the central budget, the 
departmental budget, and other funding 
sources respectively 

• A contact person and telephone number (for 
follow-up purposes) 

Data collection started in April 2000 and ended in 
mid-June 2000 after one chaser email and one 
telephone follow-up call were administered. Of the 
67 units directly involved in providing open access 


network access, only 14 did not reply or provided 
incomplete replies on the Excel spreadsheet survey 
(4 faculties and 10 departments). The other 53 replies 
included six faculties, 43 departments, the Library, 
the Computer Centre, the English Centre, and the 
Language Centre. The response rate was 79%. x The 
Finance Office provided detailed information 
concerning costs supported by the central budget 
including space, renovations, furniture, utilities, 
general maintenance, management, and so forth. 

There were a total of 873 workstations in open access 
labs in 13 units or departments on campus, providing 
a total of 108,337 computer and network access hours 
per week. Of the 43 departments that sent replies, 30 
had no computer labs. The total and average expenses 
in setting up the labs and an average setup cost per 
workstation were calculated. And finally, an average 
cost per hour of access was calculated. With the 
exception of 10% of the total setup cost and one 
percent of the total recurrent cost, costs were borne 
by the University, either from the central budget or 
faculty and department budgets. 
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Setup and recurrent costs for the notebook programme included the following: 


Setup Costs 

Recurrent Costs 

• Staffing 

. Facilities (including renovation, 

furniture and space) 

• Network upgrading 

. Administration/publicity 

• Security 

• Maintenance Repair 

• Insurance 
. Hardware 
. Software 

. Staffing 

• Facilities (including renovation, furniture 
and space) 

• Network upgrading 

• Administration/publicity 

• Security 

• Maintenance Repair 

• Insurance 

• Hardware 

• Software 


THE CHASE — SECOND DAY 
“There she blows! — she blows! — she 
blows — right ahead! ” was the masthead 
cry. “Aye, aye! ” cried Stubb, “I knew it - 
ye can ’t escape - blow on and split your 
pout, O whale! (p. 605 ) 

Study of Notebook Computer Programme 

In the case of the notebook computer programme, the 
sub-questions were: 

• How much does it cost to provide one hour of 
access to a computer and the network for 
undergraduate students through the notebook 
computer programme? 

• Who pays for what in providing access to a 
computer and the network via the notebook 
computer programme, i.e., how much of the 
costs are funded from the University’s central 
budget, faculty and departmental budgets, 
students and parents, the vendor, and other 
sources? 

The output was one hour of access to a computer by 
means of notebook computer ownership and access to 
the network by means of ACEnet and the 
University’s modem pool. The activities required to 
produce this output include the setting up and 
operation of the notebook computer programme, e.g., 
publicity, printing of flyers and handouts, the 
installation of pre-loaded software, distribution of the 
computers, and so forth. It also includes the 
installation of additional network access points for 
ACEnet, some facilities renovation (e.g., shelves in 
hallways for notebook computers near ACEnet 
access points in the Run Run Shaw Building, the 
space for the vendor's service centre), and 
maintenance of the notebook computers. 


The following academic and support units were 
involved in the implementation and maintenance of 
the notebook computer programme:™ Centre for the 
Advancement of University Teaching, Computer 
Centre, Estates Office, Faculty of Education, Finance 
Office, Library, Registry, and the Vendor (IBM). 

Heads of all involved academic and support units 
were initially contacted by email to request 
information concerning costs associated with the 
notebook computer programme. In addition, meetings 
were held with representatives from the Finance 
Office, Computer Centre, and the University Library. 
The vendor was contacted by email and telephone. 
All of the organizations contacted provided 
information for the study in a variety of ways 
including email, telephone interview, fax documents, 
and verbally and in documents collected during 
meetings. 

A total of 6,799 notebook computers were sold to 
incoming students during the three years of the 
notebook computer programme. xu These computers 
provide 1,142,232 total access hours per week; this is 
24 hours a day, 7 days a week access for participants 
from on- or off-campus. 

The total annual recurrent cost for the 6,799 notebook 
computers and an average cost per notebook 
computer were calculated. The University paid about 
55% of the setup cost and about 88% of the recurrent 
cost. Students paid about 45% of the setup cost and 
no recurrent cost. The vendor paid less than one 
percent of the setup cost and about 12% of the annual 
recurrent cost. 
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THE CHASE— THIRD DAY 
“Heart of wrought steel! ” murmured 
Starbuck gazing over the side, and 
following with his eyes the receding boat- 
-“canst thou yet ring boldly to that sight? 

— lowering thy keel among ravening 
sharks, and followed by them, open- 
mouthed to the chase; and this the critical 
third day? — For when three days flow 
together in one continuous intense 
pursuit; be sure the first is the morning, 
the second the noon, and the third the 
evening and the end of the thing — be that 
end what it may. ” (p.61 7) 

Comparing the Two Approaches 

We undertook the cost analysis study to answer three 
questions: how much do both approaches to 

providing students with access to a computer and the 
network cost, under either approach; who pays which 
costs; and finally, from the University’s perspective, 
which approach is most cost-effective? 

Any attempt to answer these questions must be 
tempered by the fact that not all of the costs 
associated with either programme are included, that 
some of the costs may be inaccurate due to 
estimations, and that some misunderstandings of 
what data were required may have resulted in 
inaccurately reported figures. Nonetheless, the 
authors are confident that the costs reported are 
reasonable approximations of the true costs and that 
missing data would not significantly change the 
results. 

How Much Does Each Approach Cost? xi:L:L 

Throughout the rest of this paper, costs are reported 
in artificially created “Resource Units”, rounded to 


the nearest tenth of a unit, instead of in actual Hong 
Kong Dollars. This ensures confidentiality of 
financial data while still allowing readers to make 
comparisons between the two approaches. As regards 
the first question, “How much does each of the two 
methods of providing open access to undergraduate 
students cost?” Table 7-1 summarizes the total and 
the University’s costs associated with each approach 
it then extrapolates the costs for a notebook computer 
programme in which it assumes that 100% of 
undergraduate students participate. 

The total setup costs for both approaches amounts to 
only about 4% of the annual University budget. xlv 
The annual recurrent costs for both programmes 
combined are about 0.8% of HKU's annual budget. xv 
If one amortizes setup costs for three years, the 
length of the current HKU/IBM Partnership 
Agreement, and adds the recurrent costs, then the 
University uses about 2% of its annual budget each 
year to provide students with computer and network 
access under both approaches combined. 

Who Pays For Which Costs? 

Under the open access laboratory approach, the 
University, i.e., the central budget, faculty or 
departmental budgets, pays 90% of the setup cost and 
99% of the recurrent cost. The remaining cost is 
funded from other means including grants, donations 
and so forth. 

Under the notebook computer approach, the 
University's share of the setup cost is about 55% and 
approximately 88% of the annual recurrent cost. 
Students pay about 45% of the setup cost and no 
recurrent cost. The vendor pays less than one percent 
of the setup cost and pays about 12% of the annual 
recurrent cost (see Table 7-2). 


Table 7-1: Programme Expenditures in Resource Units 


Approach 

Total Cost 

Total University Cost 


Setup 

Recurrent 

Setup 

Recurrent 

Labs 

24.9 

18.8 

22.3 

14.6 

Notebooks (82%) 

135.4 

5.2 

73.7 

4.6 

Notebooks (100%) 

157.9 

5.1 

84.0 

4.6 
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Table 7-2: Who Pays What Percentage of Total Cost? 



Open Access 

Laboratories 

Notebook Programme 


Setup 

Recurrent 

Setup 

Recurrent 


% 

% 

% 

% 

University 

90 

99 

55 

88 

Students 

0 

0 

45 

0 

Vendor 

0 

0 

0 

12 

Other 

10 

1 

0 

0 

Total: 

100 

100 

100 

100 


From the University's Perspective , Which 
Approach is Most Cost-Effective 

For this question, no simple answer is possible. 
Which approach is most cost-effective depends on 
how one views the data and what value is assigned to 
various issues. We attempt to draw no final 
conclusion, but rather suggest several ways of 
looking at the results of the study from which others 
may draw conclusions. 

Total Amount Spent 

First, if we consider only the total amount of money 
spent on each approach, then the open access 
laboratories would be more cost-effective because 
providing access in laboratories is less expensive. 
The notebook programme approach, if extrapolated 
to include providing 100% of undergraduate students 
with computers and network access rather than just 
the 82% who have chosen to participate in the 
voluntary programme, would be 1.5 times more 
expensive than the existing open access laboratories 
which, in theory, provide 100% of undergraduates 
with such equipment and access (albeit more 
inconveniently and for fewer hours). From this 
perspective, one can conclude that the open access 


laboratories approach is more cost effective than the 
notebook computer programme (see Table 7-3). 

Computers Purchased 

Another way to answer the question would be to ask 
what the money spent actually bought. Taking into 
account only the University’s expenses, the cost per 
machine for setup for open access labs is about 2.5 
times more expensive than the notebook programme 
with 100% participation. The annual recurrent costs 
for the open access laboratories are about 31 times 
more expensive than the notebook programme with 
100% participation. 

In the case of the open access laboratories, the 
resource units expended purchased student access to 
873 computer workstations on campus for three 
years, a 9.5:1 ratio of students to computer. In 
contrast, suppose all the students in the University 
were to acquire notebooks for three years (a total of 
8,294 notebooks — about 10 times the number of 
computers currently available in laboratories); this 
would cost only 1.5 times as much as the current 
expenditure on labs, but would provide all students 
with 24 x 7 access. From this perspective, one can 
conclude that the notebook computer programme is 
more cost effective than the open access laboratories 
approach (see Table 7-4). 


Table 7-3: Programme Expenditure in Resource Units 


Approach 

University Cost (only) 


Setup 

Recurrent 

Setup + Recurrent x 3 

Labs 

22.3 

14.6 

66.2 

Notebooks (82%) 

73.7 

4.6 

87.3 

Notebooks (100%) 

84.0 

4.6 

97.7 
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Table 7-4: Number of Computers & Student/Computer Ratio Under Each Approach 


Approach 

No. of Computers 

Student/Computer Ratio 

Open Access Laboratories 

873 

9.5:1 

Notebook Computer Programme (100%) 

8,294 

1:1 


University Costs Per Access Hour 

A third way of assessing cost-effectiveness would be 
to compare the two approaches in terms of the cost of 
providing one hour of student access to a computer 
and the network. Over three years, providing student 
access to a computer and the network in open access 
laboratories costs the University almost nine times 
(8.7) more per hour of access than under the 
notebook computer approach at 100% participation. 
From this perspective, one can conclude that the 
notebook computer programme is more cost effective 
than the open access laboratories approach. 

Other Considerations 

Apart from the question of the resources expended on 
either approach, and how best to view those 
expenditures, there are less quantitative issues that 
may be factored into any decision concerning which 
approach is most cost effective. These include that 
student costs are a significant proportion of the 
notebook programme, whereas they are non-existent 
under the laboratory approach; that alternative uses of 
the resources spent on either approach - should one 
be terminated - could provide other enhanced 
educational opportunities; that there may be added 
educational value in providing students with 24 hours 
a day, 7 days a week, computer and network access; 
and that costs associated with space on campus might 
be saved if the University adopted only one or the 
other approach. 

EPILOGUE 

“The drama's done. Why then here does 

any one step forth? — Because one did 

survive the wreck. ” (p. 624) 


We began our study mindful of the advice contained 
in the Higher Education Funding Council for 
England’s report, Information Systems and 
Technology Management: Value for Money Study , 
which notes, “most institutions surveyed do not 
aggregate their total institutional expenditure on 
IS/IT. Without such information, it is very difficult 
for institutions to assess the value (financial or 
otherwise) obtained from their investment in 
information technology, or to manage that investment 
in a fully effective way” (HEFCE, 1998, p. 14). 

We believe that we answered two of the three 
questions posed at the beginning of the study. We are 
reasonably certain that we have been able to report 
close approximations of how much is being spent on 
each open access approach and who is paying for 
what. As to the last question, which approach is most 
cost-effective from the Universities perspective, we 
think that we have provided useful information, 
which could help the University's policy-makers 
make that judgment. 

We also believe that this study will help HKU 
administrators assess the value of investments made 
in providing open access to our students. Our efforts 
to harpoon these costs began in February 2000 and 
did not fully draw to a close until our final report was 
presented in December 2000. But ultimately, it may 
be that our efforts - like Ahab - will sink beneath the 
waves without a trace as the University struggles 
with more pressing and important issues that it faces. 
In the end, perhaps the only result will be that, “the 
great shroud of the sea rolled on as it rolled five 
thousand years ago ” (p. 624). 


4 AH quotations from Moby-Dick or The Whale , Herman Melville, taken from the 1992 edition published by Penguin Books USA Inc., New York. 

" The issue of whether notebook ownership has improved students’ IT skills or the quality of their education is being dealt with in other studies. 
See http://www.hku.hk/caut/Homepage/itt/5_Reports/5_3surveys.htm for details. 

“ This distinction does not always hold. Some teaching laboratories and specialty labs are open to students at specific times for general purpose 
use. For the purposes of this study, academic units were asked to exclude laboratories used only for such “specialty” computing purposes, but to 
include laboratories that were used for such purposes but were also used to provide “open access”. For the latter, academic units were asked to 
provide the number of hours these labs were open for general use. 
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,v The study considers this question over a period of three years, 1 998-200 1 , the duration of the first Partnership agreement covering the notebook 
computer programme. 

v It could be argued that one could compare the actual student use of notebook computers, which our data from other surveys indicates averages 
around 10 hours per week, with the result of a calculation in which one determines the number of hours each lab is open times the number of 
workstations in the labs and divide that result by the number of students not participating in the notebook computer programme. But, as our 
notebook computer programme participants also use the open access labs, this would give a spurious result. 

V1 Any attempt to “average” student use may be useful in some ways, e.g., to project how many open access labs will meet most student needs 
most of the time, but may also be misleading. For example, calculations of the number of workstations necessary to meet students' needs for 
access based on such averages may lead to long queues for such access prior to paper deadlines and examinations week. 

vu Throughout this paper we discuss “costs”. But some costs are paid at one time at the beginning of a programme and others are recurrent, i.e., 
must be paid periodically over the life of a programme. The “total setup cost” described here can be considered as the total initial capital 
investment for each of the two approaches. The “total annual recurrent cost” can be interpreted as the total operational costs of the two 
approaches each year. In other words, in the case of the open access labs, the total setup costs discussed are the capital investment for setting up 
open access workstations on-campus in laboratories, and the total annual recurrent cost figures are the total costs for operating these laboratories 
each year. Likewise, in the case of the notebook programme, the total setup costs are the capital investment for providing undergraduate students 
with notebook computers and network access, and the total annual recurrent cost figures are the costs for maintaining the programme and 
operating the network infrastructure each year. 

vm Since the survey, the School of Economics (from within the Faculty of Social Sciences) and the School of Business have been merged to form 
a new Faculty. 

,x Admitting the complexity and the budgetary sensitivity of the analysis, instead of requesting the unit heads to provide a detailed breakdown of 
the information of the specific cost items stated in the last section, they were requested to only give a “lump-sum” of the setup and recurrent 
costs. 

x The Estates Office was not included in the calculation of the response rate. 

” Concurrent with the implementation of the notebook computer programme, the English Centre and the Department of Ecology and 
Biodiversity's Virtual School of Biodiversity received special funding for the development of on-line curricula. This funding was not taken as a 
part of the expenses related to the notebook computer programme because the curricula are available on-line to anyone from either a workstation 
in a laboratory or a notebook computer plugged into ACEnet. 

xn As noted earlier, the notebook computer programme is voluntary and not all students chose to participate. During the three years of the 
programme, 82% of eligible students chose to participate. This does not mean that only 82% of eligible students have a notebook computer. In the 
survey of non-participants mentioned earlier in this report, 98% said that they owned a computer prior to coming to HKU. Of these individuals, 
one percent said that they owned a notebook computer. However, these computers were not included in the study. 

XMI Throughout the rest of this paper, costs are reported in artificially created “Resource Units”, rounded to the nearest tenth of a unit, instead of in 
actual Hong Kong Dollars. This ensures confidentiality of financial data while still allowing readers to make comparisons between the two 
approaches. 

XIV Setup costs within the report are calculated as one-time costs because amortization of the various items included takes place on different 
schedules, making annual costs difficult to isolate. In reality, these costs are distributed over a number of years, amounting to much less than 3% 
of the University's annual budget. 

xv Conversely to setup costs, if hardware, network cabling, and so forth are in fact amortized over a period of time, these costs would rise but 
would still be a small percentage of the University's annual budget. 

XV1 It is wise to remember that we discuss cost-effectiveness only from the University perspective. Under the open laboratory approach, student 
costs are essentially zero and vendor costs are much lower. From the perspective of students or vendors, the answer to this question could be very 
different. 
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